Hyper Temporal Lidar with Dynamic Laser Control Using Laser Energy and Mirror Motion Models

ABSTRACT

A lidar system that includes a laser source and transmits laser pulses produced by the laser source toward range points in a field of view via a mirror that scans through a plurality of scan angles can use (1) a laser energy model to model the available energy in the laser source over time and (2) a mirror motion model to model motion of the mirror over time. Time slots for transmitting the targeted laser pulses can be identified using the mirror motion model, and a schedule for these pulses can be determined using energies predicted for the pulses at these time slots according to the laser energy model. Linking the model of mirror motion with the model of laser energy provides highly precise granularity when scheduling laser pulses targeted at specific range points in the field of view.

CROSS-REFERENCE AND PRIORITY CLAIM TO RELATED PATENT APPLICATIONS

This patent application claims priority to U.S. provisional patentapplication 63/166,475, filed Mar. 26, 2021, and entitled “HyperTemporal Lidar with Dynamic Laser Control”, the entire disclosure ofwhich is incorporated herein by reference.

This patent application is related to (1) U.S. patent application Ser.No. ______, filed this same day, and entitled “Hyper Temporal Lidar withDynamic Laser Control Using a Laser Energy Model” (said patentapplication being identified by Thompson Coburn Attorney Docket Number56976-213171), (2) U.S. patent application Ser. No. ______, filed thissame day, and entitled “Hyper Temporal Lidar with Dynamic Laser Controlfor Scan Line Shot Scheduling” (said patent application being identifiedby Thompson Coburn Attorney Docket Number 56976-213173), (3) U.S. patentapplication Ser. No. ______, filed this same day, and entitled “HyperTemporal Lidar with Dynamic Laser Control Using Safety Models” (saidpatent application being identified by Thompson Coburn Attorney DocketNumber 56976-213174), (4) U.S. patent application Ser. No. ______, filedthis same day, and entitled “Hyper Temporal Lidar with Shot Schedulingfor Variable Amplitude Scan Mirror” (said patent application beingidentified by Thompson Coburn Attorney Docket Number 56976-213175), (5)U.S. patent application Ser. No. ______, filed this same day, andentitled “Hyper Temporal Lidar with Dynamic Control of Variable EnergyLaser Source” (said patent application being identified by ThompsonCoburn Attorney Docket Number 56976-213176), (6) U.S. patent applicationSer. No. ______, filed this same day, and entitled “Hyper Temporal Lidarwith Dynamic Laser Control and Shot Order Simulation” (said patentapplication being identified by Thompson Coburn Attorney Docket Number56976-213177), (7) U.S. patent application Ser. No. ______, filed thissame day, and entitled “Hyper Temporal Lidar with Dynamic Laser ControlUsing Marker Shots” (said patent application being identified byThompson Coburn Attorney Docket Number 56976-213178), (8) U.S. patentapplication Ser. No. ______, filed this same day, and entitled “HyperTemporal Lidar with Elevation-Prioritized Shot Scheduling” (said patentapplication being identified by Thompson Coburn Attorney Docket Number56976-213179), (9) U.S. patent application Ser. No. ______, filed thissame day, and entitled “Hyper Temporal Lidar with Dynamic Laser ControlUsing Different Mirror Motion Models for Shot Scheduling and ShotFiring” (said patent application being identified by Thompson CoburnAttorney Docket Number 56976-213180), and (10) U.S. patent applicationSer. No. ______, filed this same day, and entitled “Hyper Temporal Lidarwith Detection-Based Adaptive Shot Scheduling” (said patent applicationbeing identified by Thompson Coburn Attorney Docket Number56976-213181), the entire disclosures of each of which are incorporatedherein by reference

INTRODUCTION

There is a need in the art for lidar systems that operate with lowlatency and rapid adaptation to environmental changes. This isparticularly the case for automotive applications of lidar as well asother applications where the lidar system may be moving at a high rateof speed or where there is otherwise a need for decision-making in shorttime intervals. For example, when an object of interest is detected inthe field of view for a lidar transmitter, it is desirable for the lidartransmitter to rapidly respond to this detection by firing highdensities of laser pulses at the detected object. However, as the firingrate for the lidar transmitter increases, this places pressure on theoperational capabilities of the laser source employed by the lidartransmitter because the laser source will need re-charging time.

This issue becomes particularly acute in situations where the lidartransmitter has a variable firing rate. With a variable firing rate, thelaser source's operational capabilities are not only impacted by periodsof high density firing but also periods of low density firing. As chargebuilds up in the laser source during a period where the laser source isnot fired, a need arises to ensure that the laser source does notoverheat or otherwise exceed its maximum energy limits.

The lidar transmitter may employ a laser source that uses opticalamplification to support the generation of laser pulses. Such lasersources have energy characteristics that are heavily impacted by timeand the firing rate of the laser source. These energy characteristics ofa laser source that uses optical amplification have importantoperational impacts on the lidar transmitter when the lidar transmitteris designed to operate with fast scan times and laser pulses that aretargeted on specific range points in the field of view.

As a technical solution to these problems in the art, the inventorsdisclose that a laser energy model can be used to model the availableenergy in the laser source over time. The timing schedule for laserpulses fired by the lidar transmitter can then be determined usingenergies that are predicted for the different scheduled laser pulseshots based on the laser energy model. This permits the lidartransmitter to reliably ensure at a highly granular level that eachlaser pulse shot has sufficient energy to meet operational needs,including when operating during periods of high density/high resolutionlaser pulse firing. The laser energy model is capable of modeling theenergy available for laser pulses in the laser source over very shorttime intervals as discussed in greater detail below. With such shortinterval time modeling, the laser energy modeling can be referred to asa transient laser energy model.

Furthermore, the inventors also disclose that mirror motion can bemodeled so that the system can also reliably predict where a scanningmirror is aimed within a field of view over time. This mirror motionmodel is also capable of predicting mirror motion over short timeintervals as discussed in greater detail below. In this regard, themirror motion model can also be referred to as a transient mirror motionmodel. The model of mirror motion over time can be linked with the modelof laser energy over time to provide still more granularity in thescheduling of laser pulses that are targeted at specific range points inthe field of view. Thus, a control circuit can translate a list ofarbitrarily ordered range points to be targeted with laser pulses into ashot list of laser pulses to be fired at such range points using themodeled laser energy coupled with the modeled mirror motion. In thisregard, the “shot list” can refer to a list of the range points to betargeted with laser pulses as combined with timing data that defines aschedule or sequence by which laser pulses will be fired toward suchrange points.

Through the use of such models, the lidar system can provide hypertemporal processing where laser pulses can be scheduled and fired athigh rates with high timing precision and high spatialtargeting/pointing precision. This results in a lidar system that canoperate at low latency, high frame rates, and intelligent range pointtargeting where regions of interest in the field of view can be targetedwith rapidly-fired and spatially dense laser pulse shots.

These and other features and advantages of the invention will bedescribed in greater detail below.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts an example lidar transmitter that uses a laser energymodel to schedule laser pulses.

FIG. 2A depicts an example process flow the control circuit of FIG. 1.

FIG. 2B-2D depict additional examples of lidar transmitters that use alaser energy model to schedule laser pulses.

FIG. 3 depicts an example lidar transmitter that uses a laser energymodel and a mirror motion model to schedule laser pulses.

FIGS. 4A-4D illustrate how mirror motion can be modeled for a mirrorthat scans in a resonant mode.

FIG. 4E depicts an example process flow for controllably adjusting anamplitude for mirror scanning.

FIG. 5 depicts an example process flow for the control circuit of FIG.3.

FIGS. 6A and 6B depict example process flows for shot scheduling usingthe control circuit of FIG. 3.

FIG. 7A depicts an example process flow for simulating and evaluatingdifferent shot ordering candidates based on the laser energy model andthe mirror motion model.

FIG. 7B depicts an example of how time slots in a mirror scan can berelated to the shot angles for the mirror using the mirror motion model.

FIG. 7C depicts an example process flow for simulating different shotordering candidates based on the laser energy model.

FIGS. 7D-7F depict different examples of laser energy predictionsproduced by the laser energy model with respect to different shot ordercandidates.

FIG. 8 depicts an example lidar transmitter that uses a laser energymodel and a mirror motion model to schedule laser pulses, where thecontrol circuit includes a system controller and a beam scannercontroller.

FIG. 9 depicts an example process flow for inserting marker shots into ashot list.

FIG. 10 depicts an example process flow for using an eye safety model toadjust a shot list.

FIG. 11 depicts an example lidar transmitter that uses a laser energymodel, a mirror motion model, and an eye safety model to schedule laserpulses.

FIG. 12 depicts an example process flow for simulating different shotordering candidates based on the laser energy model and eye safetymodel.

FIG. 13 depicts another example process for determining shot schedulesusing the models.

FIG. 14 depicts an example lidar system where a lidar transmitter and alidar receiver coordinate their operations with each other.

FIG. 15 depicts another example process for determining shot schedulesusing the models.

FIG. 16 illustrates how the lidar transmitter can change its firing rateto probe regions in a field of view with denser groupings of laserpulses.

FIGS. 17A-17F depict example process flows for prioritized selections ofelevations with respect to shot scheduling.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

FIG. 1 shows an example embodiment of a lidar transmitter 100 that canbe employed to support hyper temporal lidar. In an example embodiment,the lidar transmitter 100 can be deployed in a vehicle such as anautomobile. However, it should be understood that the lidar transmitter100 described herein need not be deployed in a vehicle. As used herein,“lidar”, which can also be referred to as “ladar”, refers to andencompasses any of light detection and ranging, laser radar, and laserdetection and ranging. In the example of FIG. 1, the lidar transmitter100 includes a laser source 102, a mirror subsystem 104, and a controlcircuit 106. Control circuit 106 uses a laser energy model 108 to governthe firing of laser pulses 122 by the laser source 102. Laser pulses 122transmitted by the laser source 102 are sent into the environment viamirror subsystem 104 to target various range points in a field of viewfor the lidar transmitter 100. These laser pulses 122 can beinterchangeably referred to as laser pulse shots (or more simply, asjust “shots”). The field of view will include different addressablecoordinates (e.g., {azimuth, elevation} pairs) which serve as rangepoints that can be targeted by the lidar transmitter 100 with the laserpulses 122.

In the example of FIG. 1, laser source 102 can use optical amplificationto generate the laser pulses 122 that are transmitted into the lidartransmitter's field of view via the mirror subsystem 104. In thisregard, a laser source 102 that includes an optical amplifier can bereferred to as an optical amplification laser source 102. In the exampleof FIG. 1, the optical amplification laser source 102 includes a seedlaser 114, an optical amplifier 116, and a pump laser 118. In this laserarchitecture, the seed laser 114 provides the input (signal) that isamplified to yield the transmitted laser pulse 122, while the pump laser118 provides the power (in the form of the energy deposited by the pumplaser 118 into the optical amplifier 116). So, the optical amplifier 116is fed by two inputs—the pump laser 118 (which deposits energy into theoptical amplifier 116) and the seed laser 114 (which provides the signalthat stimulates the energy in the optical amplifier 116 and inducespulse 122 to fire).

Thus, the pump laser 118, which can take the form of anelectrically-driven pump laser diode, continuously sends energy into theoptical amplifier 116. The seed laser 114, which can take the form of anelectrically-driven seed laser that includes a pulse formation networkcircuit, controls when the energy deposited by the pump laser 118 intothe optical amplifier 116 is released by the optical amplifier 116 as alaser pulse 122 for transmission. The seed laser 114 can also controlthe shape of laser pulse 122 via the pulse formation network circuit(which can drive the pump laser diode with the desired pulse shape). Theseed laser 114 also injects a small amount of (pulsed) optical energyinto the optical amplifier 116.

Given that the energy deposited in the optical amplifier 116 by the pumplaser 118 and seed laser 114 serves to seed the optical amplifier 116with energy from which the laser pulses 122 are generated, thisdeposited energy can be referred to as “seed energy” for the lasersource 102.

The optical amplifier 116 operates to generate laser pulse 122 from theenergy deposited therein by the seed laser 114 and pump laser 118 whenthe optical amplifier 116 is induced to fire the laser pulse 122 inresponse to stimulation of the energy therein by the seed laser 114. Theoptical amplifier 116 can take the form of a fiber amplifier. In such anembodiment, the laser source 102 can be referred to as a pulsed fiberlaser source. With a pulsed fiber laser source 102, the pump laser 118essentially places the dopant electrons in the fiber amplifier 116 intoan excited energy state. When it is time to fire laser pulse 122, theseed laser 114 stimulates these electrons, causing them to emit energyand collapse down to a lower (ground) state, which results in theemission of pulse 122. An example of a fiber amplifier that can be usedfor the optical amplifier 116 is a doped fiber amplifier such as anErbium-Doped Fiber Amplifier (EDFA).

It should be understood that other types of optical amplifiers can beused for the optical amplifier 116 if desired by a practitioner. Forexample, the optical amplifier 116 can take the form of a semiconductoramplifier. In contrast to a laser source that uses a fiber amplifier(where the fiber amplifier is optically pumped by pump laser 118), alaser source that uses a semiconductor amplifier can be electricallypumped. As another example, the optical amplifier 116 can take the formof a gas amplifier (e.g., a CO₂ gas amplifier). Moreover, it should beunderstood that a practitioner may choose to include a cascade ofoptical amplifiers 116 in laser source 102.

In an example embodiment, the pump laser 118 can exhibit a fixed rate ofenergy buildup (where a constant amount of energy is deposited in theoptical amplifier 116 per unit time). However, it should be understoodthat a practitioner may choose to employ a pump laser 118 that exhibitsa variable rate of energy buildup (where the amount of energy depositedin the optical amplifier 116 varies per unit time).

The laser source 102 fires laser pulses 122 in response to firingcommands 120 received from the control circuit 106. In an example wherethe laser source 102 is a pulsed fiber laser source, the firing commands120 can cause the seed laser 114 to induce pulse emissions by the fiberamplifier 116. In an example embodiment, the lidar transmitter 100employs non-steady state pulse transmissions, which means that therewill be variable timing between the commands 120 to fire the lasersource 102. In this fashion, the laser pulses 122 transmitted by thelidar transmitter 100 will be spaced in time at irregular intervals.There may be periods of relatively high densities of laser pulses 122and periods of relatively low densities of laser pulses 122. Examples oflaser vendors that provide such variable charge time control includeLuminbird and ITF. As examples, lasers that have the capacity toregulate pulse timing over timescales corresponding to preferredembodiments discussed herein and which are suitable to serve as lasersource 102 in these preferred embodiments are expected to exhibit laserwavelengths of 1.5 μm and available energies in a range of aroundhundreds of nano-Joules to around tens of micro-Joules, with timingcontrollable from hundreds of nanoseconds to tens of microseconds andwith an average power range from around 0.25 Watts to around 4 Watts.

The mirror subsystem 104 includes a mirror that is scannable to controlwhere the lidar transmitter 100 is aimed. In the example embodiment ofFIG. 1, the mirror subsystem 104 includes two mirrors—mirror 110 andmirror 112. Mirrors 110 and 112 can take the form of MEMS mirrors.However, it should be understood that a practitioner may choose toemploy different types of scannable mirrors. Mirror 110 is positionedoptically downstream from the laser source 102 and optically upstreamfrom mirror 112. In this fashion, a laser pulse 122 generated by thelaser source 102 will impact mirror 110, whereupon mirror 110 willreflect the pulse 122 onto mirror 112, whereupon mirror 112 will reflectthe pulse 122 for transmission into the environment. It should beunderstood that the outgoing pulse 122 may pass through varioustransmission optics during its propagation from mirror 112 into theenvironment.

In the example of FIG. 1, mirror 110 can scan through a plurality ofmirror scan angles to define where the lidar transmitter 100 is targetedalong a first axis. This first axis can be an X-axis so that mirror 110scans between azimuths. Mirror 112 can scan through a plurality ofmirror scan angles to define where the lidar transmitter 100 is targetedalong a second axis. The second axis can be orthogonal to the firstaxis, in which case the second axis can be a Y-axis so that mirror 112scans between elevations. The combination of mirror scan angles formirror 110 and mirror 112 will define a particular {azimuth, elevation}coordinate to which the lidar transmitter 100 is targeted. Theseazimuth, elevation pairs can be characterized as {azimuth angles,elevation angles} and/or {rows, columns} that define range points in thefield of view which can be targeted with laser pulses 122 by the lidartransmitter 100.

A practitioner may choose to control the scanning of mirrors 110 and 112using any of a number of scanning techniques. In a particularly powerfulembodiment, mirror 110 can be driven in a resonant mode according to asinusoidal signal while mirror 112 is driven in a point-to-point modeaccording to a step signal that varies as a function of the range pointsto be targeted with laser pulses 122 by the lidar transmitter 100. Inthis fashion, mirror 110 can be operated as a fast-axis mirror whilemirror 112 is operated as a slow-axis mirror. When operating in such aresonant mode, mirror 110 scans through scan angles in a sinusoidalpattern. In an example embodiment, mirror 110 can be scanned at afrequency in a range between around 100 Hz and around 20 kHz. In apreferred embodiment, mirror 110 can be scanned at a frequency in arange between around 10 kHz and around 15 kHz (e.g., around 12 kHz). Asnoted above, mirror 112 can be driven in a point-to-point mode accordingto a step signal that varies as a function of the range points to betargeted with laser pulses 122 by the lidar transmitter 100. Thus, ifthe lidar transmitter 100 is to fire a laser pulse 122 at a particularrange point having an elevation of X, then the step signal can drivemirror 112 to scan to the elevation of X. When the lidar transmitter 100is later to fire a laser pulse 122 at a particular range point having anelevation of Y, then the step signal can drive mirror 112 to scan to theelevation of Y. In this fashion, the mirror subsystem 104 canselectively target range points that are identified for targeting withlaser pulses 122. It is expected that mirror 112 will scan to newelevations at a much slower rate than mirror 110 will scan to newazimuths. As such, mirror 110 may scan back and forth at a particularelevation (e.g., left-to-right, right-to-left, and so on) several timesbefore mirror 112 scans to a new elevation. Thus, while the mirror 112is targeting a particular elevation angle, the lidar transmitter 100 mayfire a number of laser pulses 122 that target different azimuths at thatelevation while mirror 110 is scanning through different azimuth angles.U.S. Pat. Nos. 10,078,133 and 10,642,029, the entire disclosures ofwhich are incorporated herein by reference, describe examples of mirrorscan control using techniques and transmitter architectures such asthese (and others) which can be used in connection with the exampleembodiments described herein.

Control circuit 106 is arranged to coordinate the operation of the lasersource 102 and mirror subsystem 104 so that laser pulses 122 aretransmitted in a desired fashion. In this regard, the control circuit106 coordinates the firing commands 120 provided to laser source 102with the mirror control signal(s) 130 provided to the mirror subsystem104. In the example of FIG. 1, where the mirror subsystem 104 includesmirror 110 and mirror 112, the mirror control signal(s) 130 can includea first control signal that drives the scanning of mirror 110 and asecond control signal that drives the scanning of mirror 112. Any of themirror scan techniques discussed above can be used to control mirrors110 and 112. For example, mirror 110 can be driven with a sinusoidalsignal to scan mirror 110 in a resonant mode, and mirror 112 can bedriven with a step signal that varies as a function of the range pointsto be targeted with laser pulses 122 to scan mirror 112 in apoint-to-point mode.

As discussed in greater detail below, control circuit 106 can use alaser energy model 108 to determine a timing schedule for the laserpulses 122 to be transmitted from the laser source 102. This laserenergy model 108 can model the available energy within the laser source102 for producing laser pulses 122 over time in different shot schedulescenarios. By modeling laser energy in this fashion, the laser energymodel 108 helps the control circuit 106 make decisions on when the lasersource 102 should be triggered to fire laser pulses. Moreover, asdiscussed in greater detail below, the laser energy model 108 can modelthe available energy within the laser source 102 over short timeintervals (such as over time intervals in a range from 10-100nanoseconds), and such a short interval laser energy model 108 can bereferred to as a transient laser energy model 108.

Control circuit 106 can include a processor that provides thedecision-making functionality described herein. Such a processor cantake the form of a field programmable gate array (FPGA) orapplication-specific integrated circuit (ASIC) which providesparallelized hardware logic for implementing such decision-making. TheFPGA and/or ASIC (or other compute resource(s)) can be included as partof a system on a chip (SoC). However, it should be understood that otherarchitectures for control circuit 106 could be used, includingsoftware-based decision-making and/or hybrid architectures which employboth software-based and hardware-based decision-making. The processinglogic implemented by the control circuit 106 can be defined bymachine-readable code that is resident on a non-transitorymachine-readable storage medium such as memory within or available tothe control circuit 106. The code can take the form of software orfirmware that define the processing operations discussed herein for thecontrol circuit 106. This code can be downloaded onto the controlcircuit 106 using any of a number of techniques, such as a directdownload via a wired connection as well as over-the-air downloads viawireless networks, which may include secured wireless networks. As such,it should be understood that the lidar transmitter 100 can also includea network interface that is configured to receive such over-the-airdownloads and update the control circuit 106 with new software and/orfirmware. This can be particularly advantageous for adjusting the lidartransmitter 100 to changing regulatory environments with respect tocriteria such as laser dosage and the like. When using code provisionedfor over-the-air updates, the control circuit 106 can operate withunidirectional messaging to retain function safety.

Modeling Laser Energy Over Time

FIG. 2A shows an example process flow for the control circuit 106 withrespect to using the laser energy model 108 to govern the timingschedule for laser pulses 122. At step 200, the control circuit 106maintains the laser energy model 108. This step can include reading theparameters and expressions that define the laser energy model 108,discussed in greater detail below. Step 200 can also include updatingthe laser energy model 108 over time as laser pulses 122 are triggeredby the laser source 102 as discussed below.

In an example embodiment where the laser source 102 is a pulsed fiberlaser source as discussed above, the laser energy model 108 can modelthe energy behavior of the seed laser 114, pump laser 118, and fiberamplifier 116 over time as laser pulses 122 are fired. As noted above,the fired laser pulses 122 can be referred to as “shots”. For example,the laser energy model 108 can be based on the following parameters:

-   -   CE(t), which represents the combined amount of energy within the        fiber amplifier 116 at the moment when the laser pulse 122 is        fired at time t.    -   EF(t), which represents the amount of energy fired in laser        pulse 122 at time t;    -   E_(P), which represents the amount of energy deposited by the        pump laser 118 into the fiber amplifier 116 per unit of time.    -   S(t+δ), which represents the cumulative amount of seed energy        that has been deposited by the pump laser 118 and seed laser 114        into the fiber amplifier 116 over the time duration δ, where δ        represents the amount of time between the most recent laser        pulse 122 (for firing at time t) and the next laser pulse 122        (to be fired at time t+δ).    -   F(t+δ), which represents the amount of energy left behind in the        fiber amplifier 116 when the pulse 122 is fired at time t (and        is thus available for use with the next pulse 122 to be fired at        time t+δ).    -   CE(t+δ), which represents the amount of combined energy within        the fiber amplifier 116 at time t+δ (which is the sum of S(t+δ)        and F(t+δ))    -   EF(t+δ), which represents the amount of energy fired in laser        pulse 122 fired at time t+δ    -   a and b, where “a” represents a proportion of energy transferred        from the fiber amplifier 116 into the laser pulse 122 when the        laser pulse 122 is fired, where “b” represents a proportion of        energy retained in the fiber amplifier 116 after the laser pulse        122 is fired, where a+b=1

While the seed energy (S) includes both the energy deposited in thefiber amplifier 116 by the pump laser 118 and the energy deposited inthe fiber amplifier 116 by the seed laser 114, it should be understoodthat for most embodiments the energy from the seed laser 114 will bevery small relative to the energy from the pump laser 118. As such, apractitioner can choose to model the seed energy solely in terms ofenergy produced by the pump laser 118 over time. Thus, after the pulsedfiber laser source 102 fires a laser pulse at time t, the pump laser 118will begin re-supplying the fiber amplifier 116 with energy over time(in accordance with E_(P)) until the seed laser 116 is triggered at timet+δ to cause the fiber amplifier 116 to emit the next laser pulse 122using the energy left over in the fiber amplifier 116 following theprevious shot plus the new energy that has been deposited in the fiberamplifier 116 by pump laser 118 since the previous shot. As noted above,the parameters a and b model how much of the energy in the fiberamplifier 116 is transferred into the laser pulse 122 for transmissionand how much of the energy is retained by the fiber amplifier 116 foruse when generating the next laser pulse 122.

The energy behavior of pulsed fiber laser source 102 with respect to theenergy fired in laser pulses 122 in this regard can be expressed asfollows:

EF(t)=aCE(t)

F(t+δ)=bCE(t)

S(t+δ)=δE _(P)

CE(t+δ)=S(t+δ)+F(t+δ)

EF(t+δ)=aCE(t+δ)

With these relationships, the value for CE(t) can be re-expressed interms of EF(t) as follows:

${{CE}(t)} = \frac{{EF}(t)}{a}$

Furthermore, the value for F(t+δ) can be re-expressed in terms of EF(t)as follows:

${F\left( {t + \delta} \right)} = \frac{{bEF}(t)}{a}$

This means that the values for CE(t+δ) and EF(t+δ) can be re-expressedas follows:

${{CE}\left( {t + \delta} \right)} = {{\delta E_{P}} + \frac{{bEF}(t)}{a}}$${{EF}\left( {t + \delta} \right)} = {a\left( {{\delta E_{P}} + \frac{{bEF}(t)}{a}} \right)}$

And this expression for EF(t+δ) shortens to:

EF(t+δ)=aδE_(P) +bEF(t)

It can be seen, therefore, that the energy to be fired in a laser pulse122 at time t+δ in the future can be computed as a function of how muchenergy was fired in the previous laser pulse 122 at time t. Given thata, b, E_(P), and EF(t) are known values, and δ is a controllablevariable, these expressions can be used as the laser energy model 108that predicts the amount of energy fired in a laser pulse at selecttimes in the future (as well as how much energy is present in the fiberamplifier 116 at select times in the future).

While this example models the energy behavior over time for a pulsedfiber laser source 102, it should be understood that these models couldbe adjusted to reflect the energy behavior over time for other types oflaser sources.

Thus, the control circuit 106 can use the laser energy model 108 tomodel how much energy is available in the laser source 102 over time andcan be delivered in the laser pulses 122 for different time schedules oflaser pulse shots. With reference to FIG. 2A, this allows the controlcircuit 106 to determine a timing schedule for the laser pulses 122(step 202). For example, at step 202, the control circuit 106 cancompare the laser energy model 108 with various defined energyrequirements to assess how the laser pulse shots should be timed. Asexamples, the defined energy requirements can take any of a number offorms, including but not limited to (1) a minimum laser pulse energy,(2) a maximum laser pulse energy, (3) a desired laser pulse energy(which can be per targeted range point for a lidar transmitter 100 thatselectively targets range points with laser pulses 122), (4) eye safetyenergy thresholds, and/or (5) camera safety energy thresholds. Thecontrol circuit 106 can then, at step 204, generate and provide firingcommands 120 to the laser source 102 that trigger the laser source 102to generate laser pulses 122 in accordance with the determined timingschedule. Thus, if the control circuit 106 determines that laser pulsesshould be generated at times t1, t2, t3, . . . , the firing commands 120can trigger the laser source to generate laser pulses 122 at thesetimes.

A control variable that the control circuit 106 can evaluate whendetermining the timing schedule for the laser pulses is the value of δ,which controls the time interval between successive laser pulse shots.The discussion below illustrates how the choice of δ impacts the amountof energy in each laser pulse 122 according to the laser energy model108.

For example, during a period where the laser source 102 is consistentlyfired every δ units of time, the laser energy model 108 can be used topredict energy levels for the laser pulses as shown in the following toyexample.

-   -   Toy Example 1, where E_(P)=1 unit of energy; δ=1 unit of time;        the initial amount of energy stored by the fiber laser 116 is 1        unit of energy; a=0.5 and b=0.5:

Shot Number 1 2 3 4 5 Time t + 1 t + 2 t + 3 t + 4 t + 5 Seed Energyfrom Pump Laser (S) 1 1 1 1 1 Leftover Fiber Energy (F) 1 1 1 1 1Combined Energy (S + F) 2 2 2 2 2 Energy Fired (EF) 1 1 1 1 1

If the rate of firing is increased, this will impact how much energy isincluded in the laser pulses. For example, relative to Toy Example 1, ifthe firing rate is doubled (δ=0.5 units of time) (while the otherparameters are the same), the laser energy model 108 will predict theenergy levels per laser pulse 122 as follows below with Toy Example 2.

-   -   Toy Example 2, where E_(P)=1 unit of energy; δ=0.5 units of        time; the initial amount of energy stored by the fiber laser 116        is 1 unit of energy; a=0.5 and b=0.5:

Shot Number 1 2 3 4 5 Time t + 0.5 t + 1 t + 1.5 t + 2 t + 3.5 SeedEnergy from 0.5 0.5 0.5 0.5 0.5 Pump Laser (S) Leftover Fiber 1 0.750.625 0.5625 0.53125 Energy (F) Combined Energy 1.5 1.25 1.125 1.06251.03125 (S + F) Energy Fired (EF) 0.75 0.625 0.5625 0.53125 0.515625

Thus, in comparing Toy Example 1 with Toy Example 2 it can be seen thatincreasing the firing rate of the laser will decrease the amount ofenergy in the laser pulses 122. As example embodiments, the laser energymodel 108 can be used to model a minimum time interval in a rangebetween around 10 nanoseconds to around 100 nanoseconds. This timing canbe affected by both the accuracy of the clock for control circuit 106(e.g., clock skew and clock jitter) and the minimum required refreshtime for the laser source 102 after firing.

If the rate of firing is decreased relative to Toy Example 1, this willincrease how much energy is included in the laser pulses. For example,relative to Toy Example 1, if the firing rate is halved (δ=2 units oftime) (while the other parameters are the same), the laser energy model108 will predict the energy levels per laser pulse 122 as follows belowwith Toy Example 3.

-   -   Toy Example 3, where E_(P)=1 unit of energy; δ=2 units of time;        the initial amount of energy stored by the fiber laser 116 is 1        unit of energy; a=0.5 and b=0.5:

Shot Number 1 2 3 4 5 Time t + 2 t + 4 t + 6 t + 8 t + 10 Seed Energyfrom 2 2 2 2 2 Pump Laser (S) Leftover Fiber 1 1.5 1.75 1.875 1.9375Energy (F) Combined Energy 3 3.5 3.75 3.875 3.9375 (S + F) Energy Fired(EF) 1.5 1.75 1.875 1.9375 1.96875

If a practitioner wants to maintain a consistent amount of energy perlaser pulse, it can be seen that the control circuit 106 can use thelaser energy model 108 to define a timing schedule for laser pulses 122that will achieve this goal (through appropriate selection of values forδ).

For practitioners that want the lidar transmitter 100 to transmit laserpulses at varying intervals, the control circuit 106 can use the laserenergy model 108 to define a timing schedule for laser pulses 122 thatwill maintain a sufficient amount of energy per laser pulse 122 in viewof defined energy requirements relating to the laser pulses 122. Forexample, if the practitioner wants the lidar transmitter 100 to have theability to rapidly fire a sequence of laser pulses (for example, tointerrogate a target in the field of view with high resolution) whileensuring that the laser pulses in this sequence are each at or abovesome defined energy minimum, the control circuit 106 can define a timingschedule that permits such shot clustering by introducing a sufficientlylong value for δ just before firing the clustered sequence. This long δvalue will introduce a “quiet” period for the laser source 102 thatallows the energy in seed laser 114 to build up so that there issufficient available energy in the laser source 102 for the subsequentrapid fire sequence of laser pulses. As indicated by the decay patternof laser pulse energy reflected by Toy Example 2, increasing thestarting value for the seed energy (S) before entering the time periodof rapidly-fired laser pulses will make more energy available for thelaser pulses fired close in time with each other.

Toy Example 4 below shows an example shot sequence in this regard, wherethere is a desire to fire a sequence of 5 rapid laser pulses separatedby 0.25 units of time, where each laser pulse has a minimum energyrequirement of 1 unit of energy. If the laser source has just concludeda shot sequence after which time there is 1 unit of energy retained inthe fiber laser 116, the control circuit can wait 25 units of time toallow sufficient energy to build up in the seed laser 114 to achieve thedesired rapid fire sequence of 5 laser pulses 122, as reflected in thetable below.

-   -   Toy Example 4, where E_(P)=1 unit of energy; δ_(LONG)=25 units        of time; δ_(SHORT)=0.25 units of time; the initial amount of        energy stored by the fiber laser 116 is 1 unit of energy; a=0.5        and b=0.5; and the minimum pulse energy requirement is 1 unit of        energy:

Shot Number 1 2 3 4 5 Time t + 25 t + 25.25 t + 25.5 t + 25.75 t + 26Seed Energy from 25 0.25 0.25 0.25 0.25 Pump Laser (S) Leftover Fiber 113 6.625 3.4375 1.84375 Energy (F) Combined Energy 26 13.25 6.875 3.68752.09375 (S + F) Energy Fired (EF) 13 6.625 3.4375 1.84375 1.046875

This ability to leverage “quiet” periods to facilitate “busy” periods oflaser activity means that the control circuit 106 can provide highlyagile and responsive adaptation to changing circumstances in the fieldof view. For example, FIG. 16 shows an example where, during a firstscan 1600 across azimuths from left to right at a given elevation, thelaser source 102 fires 5 laser pulses 122 that are relatively evenlyspaced in time (where the laser pulses are denoted by the “X” marks onthe scan 1600). If a determination is made that an object of interest isfound at range point 1602, the control circuit 106 can operate tointerrogate the region of interest 1604 around range point 1602 with ahigher density of laser pulses on second scan 1610 across the azimuthsfrom right to left. To facilitate this high density period of rapidlyfired laser pulses within the region of interest 1604, the controlcircuit 106 can use the laser energy model 108 to determine that suchhigh density probing can be achieved by inserting a lower density period1606 of laser pulses during the time period immediately prior toscanning through the region of interest 1604. In the example of FIG. 16,this lower density period 1604 can be a quiet period where no laserpulses are fired. Such timing schedules of laser pulses can be definedfor different elevations of the scan pattern to permit high resolutionprobing of regions of interest that are detected in the field of view.

The control circuit 106 can also use the energy model 108 to ensure thatthe laser source 102 does not build up with too much energy. Forpractitioners that expect the lidar transmitter 100 to exhibit periodsof relatively infrequent laser pulse firings, it may be the case thatthe value for δ in some instances will be sufficiently long that toomuch energy will build up in the fiber amplifier 116, which can causeproblems for the laser source 102 (either due to equilibrium overheatingof the fiber amplifier 116 or non-equilibrium overheating of the fiberamplifier 116 when the seed laser 114 induces a large amount of pulseenergy to exit the fiber amplifier 116). To address this problem, thecontrol circuit 106 can insert “marker” shots that serve to bleed offenergy from the laser source 102. Thus, even though the lidartransmitter 100 may be primarily operating by transmitting laser pulses122 at specific, selected range points, these marker shots can be firedregardless of the selected list of range points to be targeted for thepurpose of preventing damage to the laser source 102. For example, ifthere is a maximum energy threshold for the laser source 102 of 25 unitsof energy, the control circuit 106 can consult the laser energy model108 to identify time periods where this maximum energy threshold wouldbe violated. When the control circuit 106 predicts that the maximumenergy threshold would be violated because the laser pulses have beentoo infrequent, the control circuit 106 can provide a firing command 120to the laser source 102 before the maximum energy threshold has beenpassed, which triggers the laser source 102 to fire the marker shot thatbleeds energy out of the laser source 102 before the laser source'senergy has gotten too high. This maximum energy threshold can be trackedand assessed in any of a number of ways depending on how the laserenergy model 108 models the various aspects of laser operation. Forexample, it can be evaluated as a maximum energy threshold for the fiberamplifier 116 if the energy model 108 tracks the energy in the fiberamplifier 116 (S+F) over time. As another example, the maximum energythreshold can be evaluated as a maximum value of the duration δ (whichwould be set to prevent an amount of seed energy (S) from beingdeposited into the fiber amplifier 116 that may cause damage when takingthe values for E_(P) and a presumed value for F into consideration.

While the toy examples above use simplified values for the modelparameters (e.g. the values for E_(P) and δ) for the purpose of ease ofexplanation, it should be understood that practitioners can selectvalues for the model parameters or otherwise adjust the model componentsto accurately reflect the characteristics and capabilities of the lasersource 102 being used. For example, the values for E_(P), a, and b canbe empirically determined from testing of a pulsed fiber laser source(or these values can be provided by a vendor of the pulsed fiber lasersource). Moreover, a minimum value for δ can also be a function of thepulsed fiber laser source 102. That is, the pulsed fiber laser sourcesavailable from different vendors may exhibit different minimum valuesfor δ, and this minimum value for δ (which reflects a maximum achievablenumber of shots per second) can be included among the vendor'sspecifications for its pulsed fiber laser source.

Furthermore, in situations where the pulsed fiber laser source 102 isexpected or observed to exhibit nonlinear behaviors, such nonlinearbehavior can be reflected in the model. As an example, it can beexpected that the pulsed fiber laser source 102 will exhibit energyinefficiencies at high power levels. In such a case, the modeling of theseed energy (S) can make use of a clipped, offset (affine) model for theenergy that gets delivered to the fiber amplifier 116 by pump laser 118for pulse generation. For example, in this case, the seed energy can bemodeled in the laser energy model 108 as:

S(t+δ)=E _(p) max(a ₁ δ+a ₀,offset)

The values for a₁, a₀, and offset can be empirically measured for thepulsed fiber laser source 102 and incorporated into the modeling ofS(t+δ) used within the laser energy model 108. It can be seen that for alinear regime, the value for a₁ would be 1, and the values for a₀ andoffset would be 0. In this case, the model for the seed energy S(t+δ)reduces to δE_(P) as discussed in the examples above.

The control circuit 106 can also update the laser energy model 108 basedon feedback that reflects the energies within the actual laser pulses122. In this fashion, laser energy model 108 can better improve ormaintain its accuracy over time. In an example embodiment, the lasersource 102 can monitor the energy within laser pulses 122 at the time offiring. This energy amount can then be reported by the laser source 102to the control circuit 106 (see 250 in FIG. 2B) for use in updating themodel 108. Thus, if the control circuit 106 detects an error between theactual laser pulse energy and the modeled pulse energy, then the controlcircuit 106 can introduce an offset or other adjustment into model 108to account for this error.

For example, it may be necessary to update the values for a and b toreflect actual operational characteristics of the laser source 102. Asnoted above, the values of a and b define how much energy is transferredfrom the fiber amplifier 116 into the laser pulse 122 when the lasersource 102 is triggered and the seed laser 114 induces the pulse 122 toexit the fiber amplifier 116. An updated value for a can be computedfrom the monitored energies in transmitted pulses 122 (PE) as follows:

a=argmin_(a)(Σ_(k=1 . . . N) |PE(t _(k)+δ_(k))−aPE(t _(k))−(1−a)δt_(k)|²)

In this expression, the values for PE represent the actual pulseenergies at the referenced times (t_(k) or t_(k)+δ_(k)). This is aregression problem and can be solved using commercial software toolssuch as those available from MATLAB, Wolfram, PTC, ANSYS, and others. Inan ideal world, the respective values for PE(t) and PE(t+δ) will be thesame as the modeled values of EF(t) and EF(t+δ), However, for a varietyof reasons, the gain factors a and b may vary due to laser efficiencyconsiderations (such as heat or aging whereby back reflections reducethe resonant efficiency in the laser cavity). Accordingly, apractitioner may find it useful to update the model 108 over time toreflect the actual operational characteristics of the laser source 102by periodically computing updated values to use for a and b.

In scenarios where the laser source 102 does not report its own actuallaser pulse energies, a practitioner can choose to include aphotodetector at or near an optical exit aperture of the lidartransmitter 100 (e.g., see photodetector 252 in FIG. 2C). Thephotodetector 252 can be used to measure the energy within thetransmitted laser pulses 122 (while allowing laser pulses 122 topropagate into the environment toward their targets), and these measuredenergy levels can be used to detect potential errors with respect to themodeled energies for the laser pulses so model 108 can be adjusted asnoted above. As another example for use in a scenario where the lasersource 102 does not report its own actual laser pulse energies, apractitioner derives laser pulse energy from return data 254 withrespect to returns from known fiducial objects in a field of view (suchas road signs which are regulated in terms of their intensity values forlight returns) (see 254 in FIG. 2D) as obtained from a point cloud 256for the lidar system. Additional details about such energy derivationsare discussed below. Thus, in such an example, the model 108 can beperiodically re-calibrated using point cloud data for returns from suchfiducials, whereby the control circuit 106 derives the laser pulseenergy that would have produced the pulse return data found in the pointcloud 256. This derived amount of laser pulse energy can then becompared with the modeled laser pulse energy for adjustment of the laserenergy model 108 as noted above.

Modeling Mirror Motion Over Time

In a particularly powerful example embodiment, the control circuit 106can also model mirror motion to predict where the mirror subsystem 104will be aimed at a given point in time. This can be especially helpfulfor lidar transmitters 100 that selectively target specific range pointsin the field of view with laser pulses 122. By coupling the modeling oflaser energy with a model of mirror motion, the control circuit 106 canset the order of specific laser pulse shots to be fired to targetedrange points with highly granular and optimized time scales. Asdiscussed in greater detail below, the mirror motion model can modelmirror motion over short time intervals (such as over time intervals ina range from 5-50 nanoseconds). Such a short interval mirror motionmodel can be referred to as a transient mirror motion model.

FIG. 3 shows an example lidar transmitter 100 where the control circuit106 uses both a laser energy model 108 and a mirror motion model 308 togovern the timing schedule for laser pulses 122.

In an example embodiment, the mirror subsystem 104 can operate asdiscussed above in connection with FIG. 1. For example, the controlcircuit 106 can (1) drive mirror 110 in a resonant mode using asinusoidal signal to scan mirror 110 across different azimuth angles and(2) drive mirror 112 in a point-to-point mode using a step signal toscan mirror 112 across different elevations, where the step signal willvary as a function of the elevations of the range points to be targetedwith laser pulses 122. Mirror 110 can be scanned as a fast-axis mirror,while mirror 112 is scanned as a slow-axis mirror. In such anembodiment, a practitioner can choose to use the mirror motion model 308to model the motion of mirror 110 as (comparatively) mirror 112 can becharacterized as effectively static for one or more scans across azimuthangles.

FIGS. 4A-4C illustrate how the motion of mirror 110 can be modeled overtime. In these examples, (1) the angle theta (θ) represents the tiltangle of mirror 110, (2) the angle phi (ϕ) represents the angle at whicha laser pulse 122 from the laser source 102 will be incident on mirror110 when mirror 110 is in a horizontal position (where θ is zerodegrees—see FIG. 4A), and (3) the angle mu (μ) represents the angle ofpulse 422 as reflected by mirror 110 relative to the horizontal positionof mirror 110. In this example, the angle μ can represent the scan angleof the mirror 110, where this scan angle can also be referred to as ashot angle for mirror 110 as angle μ corresponds to the angle at whichreflected laser pulse 122′ will be directed into the field of view iffired at that time.

FIG. 4A shows mirror 110, where mirror 110 is at “rest” with a tiltangle θ of zero degrees, which can be characterized as the horizon ofmirror 110. Laser source 102 is oriented in a fixed position so thatlaser pulses 122 will impact mirror 110 at the angle ϕ relative to thehorizontal position of mirror 110. Given the property of reflections, itshould be understood that the value of the shot angle μ will be the sameas the value of angle ϕ when the mirror 110 is horizontal (where θ=0).

FIG. 4B shows mirror 110 when it has been tilted about pivot 402 to apositive non-zero value of θ. It can be seen that the tilting of mirrorto angle θ will have the effect of steering the reflected laser pulse122′ clockwise and to the right relative to the angle of the reflectedlaser pulse 122′ in FIG. 4A (when mirror 110 was horizontal).

Mirror 110 will have a maximum tilt angle that can be referred to as theamplitude A of mirror 110. Thus, it can be understood that mirror 110will scan through its tilt angles between the values of −A (whichcorresponds to −θ_(Max)) and +A (which corresponds to +θ_(Max)). It canbe seen that the angle of reflection for the reflected laser pulse 122′relative to the actual position of mirror 110 is the sum of θ+ϕ as shownby FIG. 4B. In then follows that the value of the shot angle μ will beequal to 2θ+ϕ, as can be seen from FIG. 4B.

When driven in a resonant mode according to sinusoidal control signal,mirror 110 will change its tilt angle θ according to a cosineoscillation, where its rate of change is slowest at the ends of its scan(when it changes its direction of tilt) and fastest at the mid-point ofits scan. In an example where the mirror 110 scans between maximum tiltangles of −A to +A, the value of the angle θ as a function of time canbe expressed as:

θ=A cos(2πft)

where f represents the scan frequency of mirror 110 and t representstime. Based on this model, it can be seen that the value for θ can varyfrom A (when t=0) to 0 (when t is a value corresponding to 90 degrees ofphase (or 270 degrees of phase) to −A (when t is a value correspondingto 180 degrees of phase).

This means that the value of the shot angle μ can be expressed as afunction of time by substituting the cosine expression for θ into theexpression for the shot angle of μ=2θ+ϕ as follows:

μ=2A cos(2πft)+φ

From this expression, one can then solve for t to produce an expressionas follows:

$t = \frac{\arccos\left( \frac{\mu - \varphi}{2A} \right)}{2\pi f}$

This expression thus identifies the time t at which the scan of mirror110 will target a given shot angle μ. Thus, when the control circuit 106wants to target a shot angle of μ, the time at which mirror 110 willscan to this shot angle can be readily computed given that the valuesfor ϕ, A, and f will be known. In this fashion, the mirror motion model308 can model that shot angle as a function of time and predict the timeat which the mirror 110 will target a particular shot angle.

FIG. 4C shows mirror 110 when it has been tilted about pivot 402 to anegative non-zero value of −θ. It can be seen that the tilting of mirrorto angle −θ will have the effect of steering the reflected laser pulse122′ counterclockwise and to the left relative to the angle of thereflected laser pulse 122′ in FIG. 4A (when mirror 110 was horizontal).FIG. 4C also demonstrates a constraint for a practitioner on theselection of the value for the angle ϕ. Laser source 102 will need to bepositioned so that the angle ϕ is greater than the value of A to avoid asituation where the underside of the tilted mirror 110 occludes thelaser pulse 122 when mirror is tilted to a value of θ that is greaterthan ϕ. Furthermore, the value of the angle ϕ should not be 90° to avoida situation where the mirror 110 will reflect the laser pulse 122 backinto the laser source 102. A practitioner can thus position the lasersource 102 at a suitable angle ϕ accordingly.

FIG. 4D illustrates a translation of this relationship to how the mirror110 scans across a field of view 450. The mirror 110 will alternatelyscan in a left-to-right direction 452 and right-to-left direction 454 asmirror 110 tilts between its range of tilt angles (e.g., θ=−A through+A). For the example of FIG. 4A where the value for θ is zero, thismeans that a laser pulse fired at the untilted mirror 110 will bedirected as shown by 460 in FIG. 4D, where the laser pulse is directedtoward a range point at the mid-point of scan. The shot angle μ for this“straight ahead” gaze is ϕ as discussed above in connection with FIG.4A. As the angle θ increases from θ=0, this will cause the laser pulsesdirected by mirror 110 to scan to the right in the field of view untilthe mirror 110 tilts to the angle θ=+A. When θ=+A, mirror 110 will be atthe furthest extent of its rightward scan 452, and it will direct alaser pulse as shown by 462. The shot angle μ for this rightmost scanposition will be the value μ=2A+ϕ. From that point, the mirror 110 willbegin scanning leftward in direction 454 by reducing its tilt angle θ.The mirror 110 will once again scan through the mid-point and eventuallyreach a tilt angle of θ=−A. When θ=−A, mirror 110 will be at thefurthest extent of its leftward scan 452, and it will direct a laserpulse as shown by 464. The shot angle μ for this leftmost scan positionwill be the value μ=ϕ−2A. From that point, the mirror 110 will begintilting in the rightward direction 450 again, and the scan repeats. Asnoted above, due to the mirror motion model 308, the control circuit 106will know the time at which the mirror 110 is targeting a shot angle ofμ_(i) to direct a laser pulse as shown by 466 of FIG. 4D.

In an example embodiment, the values for +A and −A can be values in arange between +/−10 degrees and +/−20 degrees (e.g., +/−16 degrees)depending on the nature of mirror chosen as mirror 110. In an examplewhere A is 16 degrees and mirror 110 scans as discussed above inconnection with FIGS. 4A-4D, it can be understood that the angularextent of the scan for mirror 110 would be 64 degrees (or 2A from thescan mid-point in both the right and left directions for a total of 4A).

In some example embodiments, the value for A in the mirror motion model308 can be a constant value. However, some practitioners may find itdesirable to deploy a mirror 110 that exhibits an adjustable value for A(e.g., a variable amplitude mirror such as a variable amplitude MEMSmirror can serve as mirror 110). From the relationships discussed above,it can be seen that the time required to move between two shot angles isreduced when the value for amplitude A is reduced. The control circuit106 can leverage this relationship to determine whether it is desirableto adjust the amplitude of the mirror 110 before firing a sequence oflaser pulses 122. FIG. 4E shows an example process flow in this regard.At step 470, the control circuit 106 determines the settle time (ts) forchanging the amplitude from A to A′ (where A′<A). It should beunderstood that changing the mirror amplitude in this fashion willintroduce a time period where the mirror is relatively unstable, andtime will need to be provided to allow the mirror to settle down to astable position. This settling time can be empirically determined ortracked for the mirror 110, and the control circuit 106 can maintainthis settle time value as a control parameter. At step 472, the controlcircuit 106 determines the time it will take to collect a shot list dataset in a circumstance where the amplitude of the mirror is unchanged(amplitude remains A). This time can be referenced as collection timetc. This value for tc can be computed through the use of the laserenergy model 108 and mirror motion model 308 with reference to the shotsincluded in a subject shot list. At step 474, the control circuit 106determines the time it will take to collect the same shot list data setin a circumstance where the amplitude of the mirror is changed to A′.This time can be referenced as collection time tc′. This value for tc′can be computed through the use of the laser energy model 108 and mirrormotion model 308 (as adjusted in view of the reduced amplitude of A′)with reference to the shots included in the subject shot list. At step476, the control circuit compares tc with the sum of tc′ and ts. If thesum (tc′+ts) is less than tc, this means that it will be time efficientto change the mirror amplitude to A′. In this circumstance, the processflow proceeds to step 478, and the control circuit 106 adjusts theamplitude of mirror 110 to A′. If the sum (tc′+ts) is not less than tc,then the control circuit 106 leaves the amplitude value unchanged (step480).

Model-Based Shot Scheduling

FIG. 5 shows an example process flow for the control circuit 106 to useboth the laser energy model 108 and the mirror motion model 308 todetermine the timing schedule for laser pulses 122. Step 200 can operateas described above with reference to FIG. 2A to maintain the laserenergy model 108. At step 500, the control circuit 106 maintains themirror motion model 308. As discussed above, this model 308 can modelthe shot angle that the mirror will target as a function of time.Accordingly, the mirror motion model 308 can predict the shot angle ofmirror 110 at a given time t. To maintain and update the model 308, thecontrol circuit 108 can establish the values for A, ϕ, and f to be usedfor the model 308. These values can be read from memory or determinedfrom the operating parameters for the system.

At step 502, the control circuit 106 determines a timing schedule forlaser pulses 122 using the laser energy model 108 and the mirror motionmodel 308. By linking the laser energy model 108 and the mirror motionmodel 308 in this regard, the control circuit 106 can determine how muchenergy is available for laser pulses targeted toward any of the rangepoints in the scan pattern of mirror subsystem 104. For purposes ofdiscussion, we will consider an example embodiment where mirror 110scans in azimuth between a plurality of shot angles at a high rate whilemirror 112 scans in elevation at a sufficiently slower rate so that thediscussion below will assume that the elevation is held steady whilemirror 110 scans back and forth in azimuth. However, the techniquesdescribed herein can be readily extended to modeling the motion of bothmirrors 110 and 112.

If there is a desire to target a range point at a Shot Angle A with alaser pulse of at least X units of energy, the control circuit 106, atstep 502, can consult the laser energy model 108 to determine whetherthere is sufficient laser energy for the laser pulse when the mirror110's scan angle points at Shot Angle A. If there is sufficient energy,the laser pulse 122 can be fired when the mirror 110 scans to Shot AngleA. If there is insufficient energy, the control circuit 106 can wait totake the shot until after mirror 110 has scanned through and back topointing at Shot Angle A (if the laser energy model 108 indicates thereis sufficient laser energy when the mirror returns to Shot Angle A). Thecontrol circuit 106 can compare the shot energy requirements for a setof shot angles to be targeted with laser pulses to determine when thelaser pulses 122 should be fired. Upon determination of the timingschedule for the laser pulses 122, the control circuit 106 can generateand provide firing commands 120 to the laser source 102 based on thisdetermined timing schedule (step 504).

FIGS. 6A and 6B show example process flows for implementing steps 502and 504 of FIG. 5 in a scenario where the mirror subsystem 104 includesmirror 110 that scans through azimuth shot angles in a resonant mode(fast-axis) and mirror 112 that scans through elevation shot angles in apoint-to-point mode (slow-axis). Lidar transmitter 100 in these examplesseeks to fire laser pulses 122 at intelligently selected range points inthe field of view. With the example of FIG. 6A, the control circuit 106schedules shots for batches of range points at a given elevation onwhichever scan direction of the mirror 110 is schedulable for thoserange points according to the laser energy model 108. With the exampleof FIG. 6B, the control circuit 106 seeks to schedule shots for as manyrange points as it can at a given elevation for each scan direction ofthe mirror 110 in view of the laser energy model 108. For any shots atthe subject elevation that cannot be scheduled for a given scandirection due to energy model constraints, the control circuit 106 thenseeks to schedule those range points on the reverse scan (and so onuntil all of the shots are scheduled).

The process flow of FIG. 6A begins with step 600. At step 600, thecontrol circuit 106 receives a list of range points to be targeted withlaser pulses. These range points can be expressed as (azimuth angle,elevation angle) pairs, and they may be ordered arbitrarily.

At step 602, the control circuit 106 sorts the range points by elevationto yield sets of azimuth shot angles sorted by elevation. Theelevation-sorted range points can also be sorted by azimuth shot angle(e.g., where all of the shot angles at a given elevation are sorted inorder of increasing azimuth angle (smallest azimuth shot angle tolargest azimuth shot angle) or decreasing azimuth angle (largest azimuthshot angle to smallest azimuth shot angle). For the purposes ofdiscussing the process flows of FIGS. 6A and 6B, these azimuth shotangles can be referred to as the shot angles for the control circuit106. Step 602 produces a pool 650 of range points to be targeted withshots (sorted by elevation and then by shot angle).

At step 604, the control circuit 106 selects a shot elevation from amongthe shot elevations in the sorted list of range points in pool 650. Thecontrol circuit 106 can make this selection on the basis of any of anumber of criteria. The order of selection of the elevations will governwhich elevations are targeted with laser pulses 122 before others.

Accordingly, in an example embodiment, the control circuit 106 canprioritize the selection of elevations at step 604 that are expected toencompass regions of interest in the field of view. As an example, somepractitioners may find the horizon in the field of view (e.g., a roadhorizon) to be high priority for targeting with laser pulses 122. Insuch a case, step 604 can operate as shown by FIG. 17A to determine theelevation(s) which correspond to a horizon in the field of view (e.g.identify the elevations at or near the road horizon) (see step 1702) andthen prioritize the selection of those elevations from pool 650 (seestep 1702). Step 1702 can be performed by analyzing lidar return pointcloud data and/or camera images of the field of view to identify regionsin the field of view that are believed to qualify as the horizon (e.g.,using contrast detection techniques, edge detection techniques, and/orother pattern processing techniques applied to lidar or image data).

As another example, the control circuit 106 can prioritize the selectionof elevations based on the range(s) to detected object(s) in the fieldof view. Some practitioners may find it desirable to prioritize theshooting of faraway objects in the field of view. Other practitionersmay find it desirable to prioritize the shooting of nearby objects inthe field of view. Thus, in an example such as that shown by FIG. 17B,the range(s) applicable to detected object(s) is determined (see step1706). This range information will be available from the lidar returnpoint cloud data. At step 1708, the control circuit sorts the detectedobject(s) by their determined range(s). Then, at step 1708, the controlcircuit 106 prioritizes the selection of elevations from pool 650 basedon the determined range(s) for object(s) included in those elevations.With step 1708, prioritization can be given to larger range values thanfor smaller range values if the practitioner wants to shoot farawayobjects before nearby objects. For practitioners that want to shootnearby objects before faraway objects, step 1708 can give priority tosmaller range values than for larger range values. Which objects aredeemed faraway and which are deemed nearby can be controlled using anyof a number of techniques. For example, a range threshold can bedefined, and the control circuit 106 can make the elevation selectionsbased on which elevations include sorted objects whose range is above(or below as the case may be) the defined range threshold. As anotherexample, the relative ranges for the sorted objects can be used tocontrol the selection of elevations (where the sort order of eitherfarthest to nearest or nearest to farthest governs the selection ofelevations which include those objects).

As yet another example, the control circuit 106 can prioritize theselection of elevations based on the velocity(ies) of detected object(s)in the field of view. Some practitioners may find it desirable toprioritize the shooting of fast-moving objects in the field of view.FIG. 17C shows an example process flow for this. At step 1714, thevelocity is determined for each detected object in the field of view.This velocity information can be derived from the lidar return pointcloud data. At step 1716, the control circuit 106 can sort the detectedobject(s) by the determined velocity(ies). The control circuit 106 canthen use determined velocities for the sorted objects as a basis forprioritizing the selection of elevations which contain those detectedobjects (step 1718). This prioritization at step 1718 can be carried outin any of a number of ways. For example, a velocity threshold can bedefined, and step 1718 can prioritize the selection of elevation includean object moving at or above this defined velocity threshold. As anotherexample, the relative velocities of the sorted objects can be used wherean elevation that includes an object moving faster than another objectcan be selected before an elevation that includes the another (slowermoving) object.

As yet another example, the control circuit 106 can prioritize theselection of elevations based on the directional heading(s) of detectedobject(s) in the field of view. Some practitioners may find it desirableto prioritize the shooting of objects in the field of view that movingtoward the lidar transmitter 100. FIG. 17D shows an example process flowfor this. At step 1720, the directional heading is determined for eachdetected object in the field of view. This directional heading can bederived from the lidar return point cloud data. The control circuit 1722can then prioritize the selection of elevation(s) that include object(s)that are determined to be heading toward the lidar transmitter 100(within some specified degree of tolerance where the elevation thatcontains an object heading near the lidar transmitter 100 would beselected before an elevation that contains an object moving away fromthe lidar transmitter 100).

Further still, some practitioners may find it desirable to combine theprocess flows of FIG. 17C and 17D to prioritize the selection offast-moving objects that are heading toward the lidar transmitter 100.An example for this is shown by FIG. 17E. With FIG. 17E, steps 1714 and1720 can be performed as discussed above. At step 1724, the detectedobject(s) are sorted by their directional headings (relative to thelidar transmitter 100) and then by the determined velocities. At step1726, the elevations which contain objected deemed to be heading towardthe lidar transmitter 100 (and moving faster than other such objects)are prioritized for selection.

In another example embodiment, the control circuit 106 can selectelevations at step 604 based on eye safety or camera safety criteria.For example, eye safety requirements may specify that the lidartransmitter 100 should not direct more than a specified amount of energyin a specified spatial area over of a specified time period. To reducethe risk of firing too much energy into the specified spatial area, thecontrol circuit 106 can select elevations in a manner that avoidssuccessive selections of adjacent elevations (e.g., jumping fromElevation 1 to Elevation 3 rather than Elevation 2) to insert moreelevation separation between laser pulses that may be fired close intime. This manner of elevation selection may optionally be implementeddynamically (e.g., where elevation skips are introduced if the controlcircuit 106 determines that the energy in a defined spatial area hasexceeded some level that is below but approaching the eye safetythresholds). Furthermore, it should be understood that the number ofelevations to skip (a skip interval) can be a value selected by apractitioner or user to define how many elevations will be skipped whenprogressing from elevation-to-elevation. As such, a practitioner maychoose to set the elevation skip interval to be a value larger than 1(e.g., a skip interval of 5, which would cause the system to progressfrom Elevation 3 to Elevation 9). Furthermore, similar measures can betaken to avoid hitting cameras that may be located in the field of viewwith too much energy. FIG. 17F depicts an example process flow for thisapproach. At step 1730, the control circuit 106 selects Elevation X_(t)(where this selected elevation is larger (or smaller) than the precedingselected elevation (Elevation X_(t−1)) by the defined skip interval.Then, the control circuit 106 schedules the shots for the selectedelevation (step 1732), and the process flow returns to step 1730 wherethe next elevation (Elevation X_(t+1)) is selected (according to theskip interval relative to Elevation X_(t)).

Thus, it should be understood that step 604 can employ a prioritizedclassification system that decides the order in which elevations are tobe targeted with laser pulses 122 based on the criteria of FIGS. 17A-17For any combinations of any of these criteria.

At step 606, the control circuit 106 generates a mirror control signalfor mirror 112 to drive mirror 112 so that it targets the angle of theselected elevation. As noted, this mirror control signal can be a stepsignal that steps mirror 112 up (or down) to the desired elevationangle. In this fashion, it can be understood that the control circuit106 will be driving mirror 112 in a point-to-point mode where the mirrorcontrol signal for mirror 112 will vary as a function of the rangepoints to be targeted with laser pulses (and more precisely, as afunction of the order of range points to be targeted with laser pulses).

At step 608, the control circuit 106 selects a window of azimuth shotangles that are in the pool 650 at the selected elevation. The size ofthis window governs how many shot angles that the control circuit 106will order for a given batch of laser pulses 122 to be fired. Thiswindow size can be referred to as the search depth for the shotscheduling. A practitioner can configure the control circuit 106 to setthis window size based on any of a number of criteria. While the toyexamples discussed below use a window size of 3 for purposes ofillustration, it should be understood that practitioners may want to usea larger (or smaller) window size in practice. For example, in anexample embodiment, the size of the window may be a value in a rangebetween 2 shots and 12 shots. However, should the control circuit 106have larger capacities for parallel processing or should there be morelenient time constraints on latency, a practitioner may find itdesirable to choose larger window sizes. Furthermore, the controlcircuit 106 can consider a scan direction for the mirror 110 whenselecting the shot angles to include in this window. Thus, if thecontrol circuit 106 is scheduling shots for a scan directioncorresponding to increasing shot angles, the control circuit 106 canstart from the smallest shot angle in the sorted pool 650 and includeprogressively larger shot angles in the shot angle sort order of thepool 650. Similarly, if the control circuit 106 is scheduling shots fora scan direction corresponding to decreasing shot angles, the controlcircuit 106 can start from the largest shot angle in the sorted pool 650and include progressively smaller shot angles in the shot angle sortorder of the pool 650.

At step 610, the control circuit 106 determines an order for the shotangles in the selected window using the laser energy model 108 and themirror motion model 308. As discussed above, this ordering operation cancompare candidate orderings with criteria such as energy requirementsrelating to the shots to find a candidate ordering that satisfies thecriteria. Once a valid candidate ordering of shot angles is found, thiscan be used as ordered shot angles that will define the timing schedulefor the selected window of laser pulses 122. Additional details aboutexample embodiments for implementing step 610 are discussed below.

Once the shot angles in the selected window have been ordered at step610, the control circuit 106 can add these ordered shot angles to theshot list 660. As discussed in greater detail below, the shot list 660can include an ordered listing of shot angles and a scan directioncorresponding to each shot angle.

At step 612, the control circuit 106 determines whether there are anymore shot angles in pool 650 to consider at the selected elevation. Inother words, if the window size does not encompass all of the shotangles in the pool 650 at the selected elevation, then the process flowcan loop back to step 608 to grab another window of shot angles from thepool 650 for the selected elevation. If so, the process flow can thenperform steps 610 and 612 for the shot angles in this next window.

Once all of the shots have been scheduled for the shot angles at theselected elevation, the process flow can loop back from step 612 to step604 to select the next elevation from pool 650 for shot anglescheduling. As noted above, this selection can proceed in accordancewith a defined prioritization of elevations. From there, the controlcircuit 106 can perform steps 606-614 for the shot angles at the newlyselected elevation.

Meanwhile, at step 614, the control circuit 106 generates firingcommands 120 for the laser source 102 in accordance with the determinedorder of shot angles as reflected by shot list 660. By providing thesefiring commands 120 to the laser source 102, the control circuit 106triggers the laser source 102 to transmit the laser pulses 122 insynchronization with the mirrors 110 and 112 so that each laser pulse122 targets its desired range point in the field of view. Thus, if theshot list includes Shot Angles A and C to be fired at during aleft-to-right scan of the mirror 110, the control circuit 106 can usethe mirror motion model 308 to identify the times at which mirror 110will be pointing at Shot Angles A and C on a left-to-right scan andgenerate the firing commands 120 accordingly. The control circuit 106can also update the pool 650 to mark the range points corresponding tothe firing commands 120 as being “fired” to effectively remove thoserange points from the pool 650.

In the example of FIG. 6B, as noted above, the control circuit 106 seeksto schedule as many shots as possible on each scan direction of mirror110. Steps 600, 602, 604, and 606 can proceed as described above forFIG. 6A.

At step 620, the control circuit 106 selects a scan direction of mirror110 to use for scheduling. A practitioner can choose whether thisscheduling is to start with a left-to-right scan direction or aright-to-left scan direction. Then, step 608 can operate as discussedabove in connection with FIG. 6A, but where the control circuit 106 usesthe scan direction selected at step 620 to govern which shot angles areincluded in the selected window. Thus, if the selected scan directioncorresponds to increasing shot angles, the control circuit 106 can startfrom the smallest shot angle in the sorted pool 650 and includeprogressively larger shot angles in the shot angle sort order of thepool 650. Similarly, if the selected scan direction corresponds todecreasing shot angles, the control circuit 106 can start from thelargest shot angle in the sorted pool 650 and include progressivelysmaller shot angles in the shot angle sort order of the pool 650.

At step 622, the control circuit 106 determines an order for the shotangles based on the laser energy model 108 and the mirror motion model308 as discussed above for step 610, but where the control circuit 106will only schedule shot angles if the laser energy model 108 indicatesthat those shot angles are schedulable on the scan corresponding to theselected scan direction. Scheduled shot angles are added to the shotlist 660. But, if the laser energy model 108 indicates that the systemneeds to wait until the next return scan (or later) to take a shot at ashot angle in the selected window, then the scheduling of that shotangle can be deferred until the next scan direction for mirror 110 (seestep 624). This effectively returns the unscheduled shot angle to pool650 for scheduling on the next scan direction if possible.

At step 626, the control circuit 106 determines if there are any moreshot angles in pool 650 at the selected elevation that are to beconsidered for scheduling on the scan corresponding to the selected scandirection. If so, the process flow returns to step 608 to grab anotherwindow of shot angles at the selected elevation (once again taking intoconsideration the sort order of shot angles at the selected elevation inview of the selected scan direction).

Once the control circuit 106 has considered all of the shot angles atthe selected elevation for scheduling on the selected scan direction,the process flow proceeds to step 628 where a determination is made asto whether there are any more unscheduled shot angles from pool 650 atthe scheduled elevation. If so, the process flow loops back to step 620to select the next scan direction (i.e., the reverse scan direction).From there, the process flow proceeds through steps 608, 622, 624, 626,and 628 until all of the unscheduled shot angles for the selectedelevation have been scheduled and added to shot list 660. Once step 628results in a determination that all of the shot angles at the selectedelevation have been scheduled, the process flow can loop back to step604 to select the next elevation from pool 650 for shot anglescheduling. As noted above, this selection can proceed in accordancewith a defined prioritization of elevations, and the control circuit 106can perform steps 606, 620, 608, 622, 624, 626, 628, and 614 for theshot angles at the newly selected elevation.

Thus, it can be understood that the process flow of FIG. 6B will seek toschedule all of the shot angles for a given elevation during a singlescan of mirror 110 (from left-to-right or right-to-left as the case maybe) if possible in view of the laser energy model 108. However, shouldthe laser energy model 108 indicate that more time is needed to fireshots at the desired shot angles, then some of the shot angles may bescheduled for the return scan (or subsequent scan) of mirror 110.

It should also be understood that the control circuit 106 will always belistening for new range points to be targeted with new laser pulses 122.As such, steps 600 and 602 can be performed while steps 604-614 arebeing performed (for FIG. 6A) or while steps 604, 606, 620, 608, 622,624, 626, 628, and 614 are being performed (for FIG. 6B). Similarly,step 614 can be performed by the control circuit 106 while the othersteps of the FIG. 6A and 6B process flows are being performed.Furthermore, it should be understood that the process flows of FIGS. 6Aand 6B can accommodate high priority requests for range point targeting.For example, as described in U.S. Pat. No. 10,495,757, the entiredisclosure of which is incorporated herein by reference, a request maybe received to target a set of range points in a high priority manner.Thus, the control circuit 106 can also always be listening for such highpriority requests and then cause the process flow to quickly beginscheduling the firing of laser pulses toward such range points. In acircumstance where a high priority targeting request causes the controlcircuit 106 to interrupt its previous shot scheduling, the controlcircuit 106 can effectively pause the current shot schedule, schedulethe new high priority shots (using the same scheduling techniques) andthen return to the previous shot schedule once laser pulses 122 havebeen fired at the high priority targets.

Accordingly, as the process flows of FIGS. 6A and 6B work their waythrough the list of range points in pool 650, the control circuit 106will provide improved scheduling of laser pulses 122 fired at thoserange points through use of the laser energy model 108 and mirror motionmodel 308 as compared to defined criteria such as shot energy thresholdsfor those shots. Moreover, by modeling laser energy and mirror motionover short time intervals on the order of nanoseconds using transientmodels as discussed above, these shot scheduling capabilities of thesystem can be characterized as hyper temporal because highly preciseshots with highly precise energy amounts can be accurately scheduledover short time intervals if necessary.

While FIGS. 6A and 6B show their process flows as an iterated sequenceof steps, it should be understood that if the control circuit 106 hassufficient parallelized logic resources, then many of the iterations canbe unrolled and performed in parallel without the need for return loops(or using a few number of returns through the steps). For example,different windows of shot angles at the selected elevation can beprocessed in parallel with each other if the control circuit 106 hassufficient parallelized logic capacity. Similarly, the control circuit106 can also work on scheduling for different elevations at the sametime if it has sufficient parallelized logic capacity.

FIG. 7A shows an example process flow for carrying out step 610 of FIG.6A. At step 700, the control circuit 106 creates shot angle ordercandidates from the shot angles that are within the window selected atstep 608. These candidates can be created based on the mirror motionmodel 308.

For example, as shown by FIG. 7B, the times at which the mirror 110 willtarget the different potential shot angles can be predicted using themirror motion model 308. Thus, each shot angle can be assigned a timeslot 710 with respect to the scan of mirror 110 across azimuth angles(and back). As shown by FIG. 7B, if mirror 110 starts at Angle Zero atTime 1, it will then scan to Angle A at Time 2, then scan to Angle B atTime 3, and so on through its full range of angles (which in the exampleof FIG. 7B reaches Angle J before the mirror 110 begins scanning backtoward Angle Zero). The time slots for these different angles can becomputed using the mirror motion model 308. Thus, if the window of shotangles identifies Angle A, Angle C, and Angle I as the shot angles, thenthe control circuit 106 will know which time slots of the mirror scanfor mirror 110 will target those shot angles. For example, according toFIG. 7B, Time Slots 1, 3, and 9 will target Angles A, C, and I. On thereturn scan, Time Slot 11 will also target Angle I (as shown by FIG.7B), while Time Slots 17 and 19 will also target Angles C and Arespectively. As example embodiments, the time slots 710 can correspondto time intervals in a range between around 5 nanoseconds and around 50nanoseconds, which would correspond to angular intervals of around 0.01to 0.1 degrees if mirror 110 is scanning at 12 kHz over an angularextent of 64 degrees (where +/−A is +/−16 degrees).

To create the order candidates at step 700, the control circuit 106 cangenerate different permutations of time slot sequences for differentorders of the shot angles in the selected window. Continuing with anexample where the shot angles are A, C, and I, step 700 can produce thefollowing set of example order candidates (where each order candidatecan be represented by a time slot sequence):

Order Time Slot Candidate Sequence Comments Candidate 1 1, 3, 9 Thiswould correspond to firing laser pulses in the shot angle order of ACIduring the first scan for mirror 110 (which moves from left-to-right)Candidate 2 1, 9, 17 This would correspond to firing laser pulses in theshot angle order of AIC, where laser pulses are fired at Shot Angles Aand I during the first scan for mirror 110 and where the laser pulse isfired at Shot Angle C during the second (return) scan for mirror 110(where this second scan moves from right-to-left). Candidate 3 3, 9, 19This would correspond to firing laser pulses in the shot angle order ofCIA, where laser pulses are fired at Shot Angles C and I during thefirst scan for mirror 110 and where the laser pulse is fired at ShotAngle A during the second (return) scan for mirror 110. Candidate 4 3,9, 21 This would correspond to firing laser pulses in the shot angleorder of CIA, where laser pulses are fired at Shot Angles C and I duringthe first scan for mirror 110 and where the laser pulse is fired at ShotAngle A during the third scan for mirror 110 (which moves fromleft-to-right) . . . . . . . . .

It should be understood that the control circuit 106 could createadditional candidate orderings from different permutations of time slotsequences for Shot Angles A, C, and I. A practitioner can choose tocontrol how many of such candidates will be considered by the controlcircuit 106.

At step 702, the control circuit 106 simulates the performance of thedifferent order candidates using the laser energy model 108 and thedefined shot requirements. As discussed above, these shot requirementsmay include requirements such as minimum energy thresholds for eachlaser pulse (which may be different for each shot angle), maximum energythresholds for each laser pulse (or for the laser source), and/ordesired energy levels for each laser pulse (which may be different foreach shot angle).

To reduce computational latency, this simulation and comparison withshot requirements can be performed in parallel for a plurality of thedifferent order candidates using parallelized logic resources of thecontrol circuit 106. An example of such parallelized implementation ofstep 702 is shown by FIG. 7C. In the example of FIG. 7C, steps 720, 722,and 724 are performed in parallel with respect to a plurality of thedifferent time slot sequences that serve as the order candidates. Thus,steps 720 a, 722 a, and 724 a are performed for Time Slot Sequence 1;steps 720 b, 722 b, and 724 b are performed for Time Slot Sequence 2;and so on through steps 720 n, 722 n, and 724 n for Time Slot Sequencen.

At step 720, the control circuit 106 uses the laser energy model 108 topredict the energy characteristics of the laser source and resultantlaser pulse if laser pulse shots are fired at the time slotscorresponding to the subject time slot sequence. These modeled energiescan then be compared to criteria such as a maximum laser energythreshold and a minimum laser energy threshold to determine if the timeslot sequence would be a valid sequence in view of the systemrequirements. At step 722, the control circuit 106 can label each testedtime slot sequence as valid or invalid based on this comparison betweenthe modeled energy levels and the defined energy requirements. At step724, the control circuit 106 can compute the elapsed time that would beneeded to fire all of the laser pulses for each valid time slotsequence. For example, Candidate 1 from the example above would have anelapsed time duration of 9 units of time, while Candidate 2 from theexample above would have an elapsed time duration of 17 units of time.

FIGS. 7D, 7E, and 7F show examples of such simulations of time slotsequences for our example where the shot angles to be scheduled withlaser pulses are Shot Angles A, C, and I. In this scenario, we willassume that the laser energy model 108 will employ (1) the value forE_(S) as a constant value of 1 unit of energy per unit of time and (2)the values for a and b as 0.5 each. Furthermore, we will assume thatthere are 3 units of energy left in the fiber laser 116 when the scanbegins (and where the scan begins at Angle Zero while moving fromleft-to-right).

Moreover, for the purposes of this example, the energy requirements forthe shots can be defined as (8, 3, 4) for minimum shot energies withrespect to shot angles A, C, and I respectively, and where the maximumlaser energy for the laser source can be defined as 20 units of combinedseed and stored fiber energy (which would translate to a maximum laserpulse energy of 10 units of energy).

FIG. 7D shows an example result for simulating the time slot sequence oflaser pulses at time slots 1, 3, and 9. In this example, it can be seenthat this time slot sequence is invalid because the shot energy for TimeSlot 1 (targeting Shot Angle A) is only 2 units of energy, which isbelow the minimum energy threshold of 8 units for Shot Angle A. Thistime slot sequence also fails because the shot energy for Time Slot 3(targeting Shot Angle C) is only 2 units of energy, which is below theminimum energy threshold of 3 units for Shot Angle C.

FIG. 7E shows an example result for simulating the time slot sequence oflaser pulses at time slots 1, 9, and 17. In this example, it can be seenthat this time slot sequence is invalid because the shot energy for TimeSlot 1 (targeting Shot Angle A) is too low.

FIG. 7F shows an example result for simulating the time slot sequence oflaser pulses at time slots 3, 9, and 21. In this example, it can be seenthat this time slot sequence is valid because the shot energies for eachtime slot are at or above the minimum energy thresholds for theircorresponding shot angles (and none of the time slots would violate themaximum energy threshold for the laser source). It can be furthersurmised from FIG. 7F that a simulation of a Time Slot Sequence of (3,9, 19) also would have failed because there is insufficient energy in alaser pulse that would have been fired at Shot Angle A.

Accordingly, the simulation of these time slot sequences would result ina determination that the time slot sequence of (3, 9, 21) is a validcandidate, which means that this time slot sequence can define thetiming schedule for laser pulses fired toward the shot angles in theselected window. The elapsed time for this valid candidate is 21 unitsof time.

Returning to FIG. 7A, at step 704, the control circuit 106 selects thevalid order candidate which has the lowest elapsed time. Thus, in ascenario where the simulations at step 702 would have produced two ormore valid order candidates, the control circuit 106 will select theorder candidate that will complete its firing of laser pulses thesoonest which helps improve the latency of the system.

For example embodiments, the latency with which the control circuit 106is able to determine the shot angle order and generate appropriatefiring commands is an important operational characteristic for the lidartransmitter 100. To maintain high frame rates, it is desirable for thecontrol circuit 106 to carry out the scheduling operations for all ofthe shot angles at a selected elevation in the amount of time it takesto scan mirror 110 through a full left-to-right or right-to-left scan iffeasible in view of the laser energy model 108 (where this time amountis around 40 microseconds for a 12 kHz scan frequency). Moreover, it isalso desirable for the control circuit 106 to be able to schedule shotsfor a target that is detected based on returns from shots on the currentscan line during the next return scan (e.g., when a laser pulse 122fired during the current scan detects something of interest that is tobe interrogated with additional shots (see FIG. 16 discussed above)). Inthis circumstance, the detection path for a pulse return through a lidarreceiver and into a lidar point cloud generator where the target ofinterest is detected will also need to be taken into account. Thisportion of the processing is expected to require around 0.4 to 10microseconds, which leaves around 30 microseconds for the controlcircuit 106 to schedule the new shots at the region of interest duringthe next return scan if possible. For a processor of the control circuit106 which has 2 Gflops of processing per second (which is a valueavailable from numerous FPGA and ASIC vendors), this amounts to 50operations per update, which is sufficient for the operations describedherein. For example, the control circuit 106 can maintain lookup tables(LUTs) that contain pre-computed values of shot energies for differenttime slots within the scan. Thus, the simulations of step 702 can bedriven by looking up precomputed shot energy values for the defined shotangles/time slots. The use of parallelized logic by the control circuit106 to accelerate the simulations helps contribute to the achievement ofsuch low latency. Furthermore, practitioners can adjust operationalparameters such as the window size (search depth) in a manner to achievedesired latency targets.

FIG. 8 shows an example embodiment for the lidar transmitter 100 wherethe control circuit 106 comprises a system controller 800 and a beamscanner controller 802. System controller 800 and beam scannercontroller 802 can each include a processor and memory for use incarrying out its tasks. The mirror subsystem 104 can be part of beamscanner 810 (which can also be referred to as a lidar scanner). Beamscanner controller 802 can be embedded as part of the beam scanner 810.In this example, the system controller 800 can carry out steps 600, 602,604, 608, 610, and 612 of FIG. 6A if the control circuit 106 employs theFIG. 6A process flow (or steps 600, 602, 604, 620, 608, 622, 624, 626,and 628 of FIG. 6B if the control circuit 106 employs the FIG. 6Bprocess flow), while beam scanner controller 802 carries out steps 606and 614 for the FIGS. 6A and 6B process flows. Accordingly, once thesystem controller 800 has selected the elevation and the order of shotangles, this information can be communicated from the system controller800 to the beam scanner controller 802 as shot elevation 820 and orderedshot angles 822.

The ordered shot angles 822 can also include flags that indicate thescan direction for which the shot is to be taken at each shot angle.This scan direction flag will also allow the system to recognizescenarios where the energy model indicates there is a need to pass by atime slot for a shot angle without firing a shot and then firing theshot when the scan returns to that shot angle in a subsequent time slot.For example, with reference to the example above, the scan directionflag will permit the system to distinguish between Candidate 3 (for thesequence of shot angles CIA at time slots 3, 9, and 19) versus Candidate4 (for the same sequence of shot angles CIA but at time slots 3, 9, and21). A practitioner can explicitly assign a scan direction to eachordered shot angle by adding the scan direction flag to each orderedshot angle if desired, or a practitioner indirectly assign a scandirection to each ordered shot angle by adding the scan direction flagto the ordered shot angles for which there is a change in scandirection. Together, the shot elevations 802 and order shot angles 822serve as portions of the shot list 660 used by the lidar transmitter 100to target range points with laser pulses 122.

The beam scanner controller 802 can generate control signal 806 formirror 112 based on the defined shot elevation 820 to drive mirror 112to a scan angle that targets the elevation defined by 820. Meanwhile,the control signal 804 for mirror 110 will continue to be the sinusoidalsignal that drives mirror 110 in a resonant mode. However, somepractitioners may choose to also vary control signal 804 as a functionof the ordered shot angles 822 (e.g., by varying amplitude A asdiscussed above).

In the example of FIG. 8, the mirror motion model 308 can comprise afirst mirror motion model 808 a maintained and used by the beam scannercontroller 802 and a second mirror motion model 808 b maintained andused by the system controller 800. With FIG. 8, the task of generatingthe firing commands 120 can be performed by the beam scanner controller802. The beam scanner controller 810 can include a feedback system 850that tracks the actual mirror tilt angles θ for mirror 110. Thisfeedback system 850 permits the beam scanner controller 802 to closelymonitor the actual tilt angles of mirror 110 over time which thentranslates to the actual scan angles μ of mirror 110. This knowledge canthen be used to adjust and update mirror motion model 808 a maintainedby the beam scanner controller 802. Because model 808 a will closelymatch the actual scan angles for mirror 110 due to the feedback from850, we can refer to model 808 a as the “fine” mirror motion model 808a. In this fashion, when the beam scanner controller 802 is notified ofthe ordered shot angles 822 to be targeted with laser pulses 122, thebeam scanner controller 802 can use this “fine” mirror motion model 808a to determine when the mirror has hit the time slots which target theordered shot angles 822. When these time slots are hit according to the“fine” mirror motion model 808 a, the beam scanner controller 802 cangenerate and provide corresponding firing commands 120 to the lasersource 102.

Examples of techniques that can be used for the scan tracking feedbacksystem 850 are described in the above-referenced and incorporated U.S.Pat. No. 10,078,133. For example, the feedback system 850 can employoptical feedback techniques or capacitive feedback techniques to monitorand adjust the scanning (and modeling) of mirror 110. Based oninformation from the feedback system 850, the beam scanner controller802 can determine how the actual mirror scan angles may differ from themodeled mirror scan angles in terms of frequency, phase, and/or maximumamplitude. Accordingly, the beam scanner controller 802 can thenincorporate one or more offsets or other adjustments relating thedetected errors in frequency, phase, and/or maximum amplitude into themirror motion model 808 a so that model 808 a more closely reflectsreality. This allows the beam scanner controller 802 to generate firingcommands 120 for the laser source 102 that closely match up with theactual shot angles to be targeted with the laser pulses 122.

Errors in frequency and maximum amplitude within the mirror motion model808 a can be readily derived from the tracked actual values for the tiltangle θ as the maximum amplitude A should be the maximum actual valuefor θ, and the actual frequency is measurable based on tracking the timeit takes to progress from actual values for A to −A and back.

Phased locked loops (or techniques such as PID control, both availableas software tools in MATLAB) can be used to track and adjust the phaseof the model 808 a as appropriate. The expression for the tilt angle θthat includes a phase component (p) can be given as:

θ=A cos(2πft+p)

From this, we can recover the value for the phase p by the relation:

θ≈A cos(2πft)−A sin(2πft)p

Solving for p, this yields the expression:

$p = \frac{{A\cos\left( {2\pi ft} \right)} - \theta}{A\sin\left( {2\pi ft} \right)}$

Given that the tracked values for A, f, t, and θ are each known, thevalue for p can be readily computed. It should be understood that thisexpression for p assumes that the value of the p is small, which will bean accurate assumption if the actual values for A, f, t, and θ areupdated frequently and the phase is also updated frequently. Thiscomputed value of p can then be used by the “fine” mirror motion model808 a to closely track the actual shot angles for mirror 110, andidentify the time slots that correspond to those shot angles accordingto the expression:

$t = \frac{{\arccos\left( \frac{\mu - \varphi}{2A} \right)} - p}{2\pi f}$

While a practitioner will find it desirable for the beam scannercontroller 802 to rely on the highly accurate “fine” mirror motion model808 a when deciding when the firing commands 120 are to be generated,the practitioner may also find that the shot scheduling operations cansuffice with less accurate mirror motion modeling. Accordingly, thesystem controller 800 can maintain its own model 808 b, and this model808 b can be less accurate than model 808 a as small inaccuracies in themodel 808 b will not materially affect the energy modeling used todecide on the ordered shot angles 822. In this regard, model 808 b canbe referred to as a “coarse” mirror motion model 808 b. If desired, apractitioner can further communicate feedback from the beam scannercontroller 802 to the system controller 800 so the system controller 800can also adjusts its model 808 b to reflect the updates made to model808 a. In such a circumstance, the practitioner can also decide on howfrequently the system will pass these updates from model 808 a to model808 b.

Marker Shots to Bleed Off and/or Regulate Shot Energy

FIG. 9 depicts an example process flow for execution by the controlcircuit 106 to insert marker shots into the shot list in order to bleedoff energy from the laser source 102 when needed. As discussed above,the control circuit 106 can consult the laser energy model 108 asapplied to the range points to be targeted with laser pulses 122 todetermine whether a laser energy threshold would be violated. If so, thecontrol circuit 106 may insert a marker shot into the shot list to bleedenergy out of the laser source 102 (step 902). In an example embodiment,this threshold can be set to define a maximum or peak laser energythreshold so as to avoid damage to the laser source 102. In anotherexample embodiment, this threshold can be set to achieve a desiredconsistency, smoothness, and/or balance in the energies of the laserpulse shots.

For example, one or more marker shots can be fired to bleed off energyso that a later targeted laser pulse shot (or set of targeted shots)exhibits a desired amount of energy. As an example embodiment, themarker shots can be used to bleed off energy so that the targeted laserpulse shots exhibit consistent energy levels despite a variable rate offiring for the targeted laser pulse shots (e.g., so that the targetedlaser pulse shots will exhibit X units of energy (plus or minus sometolerance) even if those targeted laser pulse shots are irregularlyspaced in time). The control circuit 106 can consult the laser energymodel 108 to determine when such marker shots should be fired toregulate the targeted laser pulse shots in this manner.

Modeling Eye and Camera Safety Over Time

FIG. 10 depicts an example process flow for execution by the controlcircuit 106 where eye safety requirements are also used to define oradjust the shot list. To support these operations, the control circuit106 can also, at step 1000, maintain an eye safety model 1002. Eyesafety requirements for a lidar transmitter 100 may be established todefine a maximum amount of energy that can be delivered within a definedspatial area in the field of view over a defined time period. Since thesystem is able to model per pulse laser energy with respect to preciselytargeted range points over highly granular time periods, this allows thecontrol circuit 106 to also monitor whether a shot list portion wouldviolate eye safety requirements. Thus, the eye safety model 1002 canmodel how much aggregated laser energy is delivered to the definedspatial area over the defined time period based on the modeling producedfrom the laser energy model 108 and the mirror motion model 308. At step1010, the control circuit 106 uses the eye safety model 1002 todetermine whether the modeled laser energy that would result from asimulated sequence of shots would violate the eye safety requirements.If so, the control circuit can adjust the shot list to comply with theeye safety requirements (e.g., by inserting longer delays betweenordered shots delivered close in space, by re-ordering the shots, etc.)

FIG. 11 shows an example lidar transmitter 100 that is similar in natureto the example of FIG. 8, but where the system controller 800 alsoconsiders the eye safety model 1002 when deciding on how to order theshot angles. FIG. 12 shows how the simulation step 702 from FIG. 7A canbe performed in example embodiments where the eye safety model 1002 isused. As shown by FIG. 12, each parallel path can include steps 720,722, and 724 as discussed above. Each parallel path can also include astep 1200 to be performed prior to step 722 where the control circuit106 uses the eye safety model 1002 to test whether the modeled laserenergy for the subject time slot sequence would violate eye safetyrequirements. If the subject time slot sequence complies with thecriteria tested at steps 720 and 1200, then the subject time slotsequence can be labeled as valid. If the subject time slot sequenceviolates the criteria tested at steps 720 or 1200, then the subject timeslot sequence can be labeled as invalid.

Similar to the techniques described for eye safety in connection withFigured 10, 11, and 12, it should be understood that a practitioner canalso use the control circuit to model and evaluate whether time slotsequences would violate defined camera safety requirements. To reducethe risk of laser pulses 122 impacting on and damaging cameras in thefield of view, the control circuit can also employ a camera safety modelin a similar manner and toward similar ends as the eye safety model1002. In the camera safety scenario, the control circuit 106 can respondto detections of objects classified as cameras in the field of view bymonitoring how much aggregated laser energy will impact that cameraobject over time. If the model indicates that the camera object wouldhave too much laser energy incident on it in too short of a time period,the control circuit can adjust the shot list as appropriate.

Moreover, as noted above with respect to the laser energy model 108 andthe mirror motion model 308, the eye safety and camera safety models cantrack aggregated energy delivered to defined spatial areas over definedtime periods over short time intervals, and such short interval eyesafety and camera safety models can be referred to as transient eyesafety and camera safety models.

Additional Example Embodiments

FIG. 13 shows another example of a process flow for the control circuit106 with respect to using the models to dynamically determine the shotlist for the transmitter 100.

At step 1300, the laser energy model 108 and mirror motion model 308 areestablished. This can include determining from factory or calibrationthe values to be used in the models for parameters such as E_(P), a, b,and A. Step 1300 can also include establishing the eye safety model 1002by defining values for parameters that govern such a model (e.g.parameters indicative of limits for aggregated energy for a definedspatial area over a defined time period). At step 1302, the control lawfor the system is connected to the models established at step 1300.

At step 1304, the seed energy model used by the laser energy model 108is adjusted to account for nonlinearities. This can employ the clipped,offset (affine) model for seed energy as discussed above.

At step 1306, the laser energy model 108 can be updated based on lidarreturn data and other feedback from the system. For example, as notedabove in connection with FIG. 2D, the actual energies in laser pulses122 can be derived from the pulse return data included in point cloud256. For example, the pulse return energy can be modeled as a functionof the transmitted pulse energy according to the following expression(for returns from objects that are equal to or exceed the laser spotsize and assuming modest atmospheric attenuation):

${{Pulse}{Return}{Energy}} = {\left( \frac{{PEAperture}_{Receiver}}{\pi R^{2}} \right){Reflectivity}}$

In this expression, Pulse Return Energy represents the energy of thepulse return (which is known from the point cloud 256), PE representsthe unknown energy of the transmitted laser pulse 122,Aperture_(Receiver) represents the known aperture of the lidar receiver(see 1400 in FIG. 14), R represents the measured range for the return(which is known from the point cloud 256), and Reflectivity representsthe percentage of reflectivity for the object from which the return wasreceived. Therefore, one can solve for PE so long as the reflectivity isknown. This will be the case for objects like road signs whosereflectivity is governed by regulatory agencies. Accordingly, by usingreturns from known fiducials such as road signs, the control circuit 106can derive the actual energy of the transmitted laser pulse 122 and usethis value to facilitate determinations as to whether any adjustments tothe laser energy model 108 are needed (e.g., see discussions above reupdating the values for a and b based on PE values which represent theactual energies of the transmitted laser pulses 122).

Also, at step 1308, the laser health can be assessed and monitored as abackground task. The information derived from the feedback received forsteps 1306 and 1308 can be used to update model parameters as discussedabove. For example, as noted above, the values for the seed energy modelparameters as well as the values for a and b can be updated by measuringthe energy produced by the laser source 102 and fitting the data to theparameters. Techniques which can be used for this process include leastsquares, sample matrix inversion, regression, and multiple exponentialextensions. Further still, as noted above, the amount of error can bereduced by using known targets with a given reflectivity and using theseto calibrate the system. This is helpful because the reflectivity of aquantity that is known, i.e. a fiducial, allows one to explicitlyextract shot energy (after backing out range dependencies and anyobliquity). Examples of fiducials that may be employed include roadsigns and license plates.

At step 1310, the lidar return data and the coupled models can be usedto ensure that the laser pulse energy does not exceed safety levels.These safety levels can include eye safety as well as camera safety asdiscussed above. Without step 1310, the system may need to employ a muchmore stringent energy requirement using trial and error to establishlaser settings to ensure safety. For example if we only had a lasermodel where the shot energy is accurate to only ±3 J per shot around thepredicted shot, and maximum shot energy is limited to 8, we could notuse any shots predicted to exceed 5. However, the hyper temporalmodeling and control that is available from the laser energy model 108and mirror motion model 308 as discussed herein allows us to obtainaccurate predictions within a few percent error, virtually erasing theoperational lidar impact of margin.

At step 1312, the coupled models are used with different orderings ofshots, thereby obtaining a predicted shot energy in any chosen orderedsequence of shots drawn from the specified list of range points. Step1312 may employ simulations to predict shot energies for different timeslots of shots as discussed above.

At step 1314, the system inserts marker shots in the timing schedule ifthe models predict that too much energy will build up in the lasersource 102 for a given shot sequence. This reduces the risk of too muchenergy being transferred into the fiber laser 116 and causing damage tothe fiber laser 116.

At step 1316, the system determines the shot energy that is needed todetect targets with each shot. These values can be specified as aminimum energy threshold for each shot. The value for such threshold(s)can be determined from radiometric modeling of the lidar, and theassumed range and reflectivity of a candidate target. In general, thisstep can be a combination of modeling assumptions as well asmeasurements. For example, we may have already detected a target, so thesystem may already know the range (within some tolerance). Since theenergy required for detection is expected to vary as the square of therange, this knowledge would permit the system to establish the minimumpulse energy thresholds so that there will be sufficient energy in theshots to detect the targets.

Steps 1318 and 1320 operate to prune the candidate ordering optionsbased on the energy requirements (e.g., minimum energy thresholds pershot) (for step 1318) and shot list firing completion times (to favorvalid candidate orderings with faster completion times) (for step 1320).

At step 1322, candidate orderings are formed using elevation movementson both scan directions. This allows the system to consider taking shotson both a left-to-right scan and a right-to-left scan. For example,suppose that the range point list has been completed on a certainelevation, when the mirror is close to the left hand side. Then it isfaster to move the elevation mirror at that point in time and begin thefresh window of range points to be scheduled beginning on this same lefthand side and moving right. Conversely, if we deplete the range pointlist when the mirror is closer to the right hand side it is faster tomove the mirror in elevation whilst it is on the right hand side.Moreover, in choosing an order from among the order candidates, and whenmoving from one elevation to another, movement on either side of themirror motion, the system may move to a new elevation when mirror 110 isat one of its scan extremes (full left or full right). However, ininstances where a benefit may arise from changing elevations when mirror110 is not at one of its scan extremes, the system may implementinterline skipping as described in the above-referenced and incorporatedU.S. Pat. No. 10,078,133. The mirror motion model 308 can also beadjusted to accommodate potential elevation shift during a horizontalscan.

At step 1324, if processing time allows the control circuit 106 toimplement auctioning (whereby multiple order candidates areinvestigated, the lowest “cost” (e.g., fastest lidar execution time)order candidate is selected by the control circuit 106 (acting as“auctioneer”). A practitioner may not want the control circuit toconsider all of the possible order candidates as this may be toocomputationally expensive and introduce an undue amount of latency.Thus, the control circuit 106 can enforce maximums or other controls onhow many order candidates are considered per batch of shots to beordered. Greedy algorithms can be used when choosing ordering shots.Generally, the system can use a search depth value (which defines howmany shots ahead the control circuit will evaluate) in this process in amanner that is consistent with any real time consideration in shot listgeneration. At step 1326, delays can be added in the shot sequence tosuppress a set of shots and thus increase available shot energy toenable a finer (denser) grid as discussed above. The methodology forsorting through different order candidates can be considered a specialcase of the Viterbi algorithm which can be implemented using availablesoftware packages such as Mathworks. This can also be inferred usingequivalence classes or group theoretic methods. Furthermore, if thesystem detects that reduced latency is needed, the search depth can bereduced (see step 1328).

FIG. 14 depicts an example embodiment for a lidar transmitter 100 thatshows how the system controller 800 can interact with the lidar receiver1400 to coordinate system operations. The lidar receiver 1400 canreceive and process pulse returns 1402 to compute range information forobjects in the field of view impacted by the laser pulses 122. Thisrange information can then be included in the point cloud 1404 generatedby the lidar system. Examples of suitable technology for use as thelidar receiver 1400 are described in U.S. Pat. Nos. 9,933,513 and10,754,015, the entire disclosures of which are incorporated herein byreference. In the example of FIG. 14, the system controller 800 can usethe point cloud 1404 to intelligently select range points for targetingwith laser pulses, as discussed in the above-referenced and incorporatedpatents. For example, the point cloud data can be used to determineranges for objects in the field of view that are to be targeted withlaser pulses 122. The control circuit 106 can use this range informationto determine desired energy levels for the laser pulses 122 which willtarget range points that are believed to correspond to those objects. Inthis fashion, the control circuit 106 can controllably adjust the laserpulse energy as a function of the estimated range of the object beingtargeted so the object is illuminated with a sufficient amount of lightenergy given its estimated range to facilitate adequate detection by thelidar receiver 1400. Further still, the beam scanner controller 802 canprovide shot timing information 1410 to the receiver 1400 and the systemcontroller 800 can provide shot data 1412 (such as data identifying thetargeting range points) to the receiver 1400. The combination of thisinformation informs the receiver how to control which pixels of thereceiver 1400 should be activated for detecting pulse returns 1402(including when those pixels should be activated). As discussed in theabove-referenced and incorporated '513 and '015 patents, the receivercan select pixels for activation to detect pulse returns 1402 based onthe locations of the targeted range points in the field of view.Accordingly, precise knowledge of which range points were targeted andwhen those range points were targeted helps improve the operations ofreceiver 1400. Although not shown in FIG. 14, it should also beunderstood that a practitioner may choose to also include a camera thatimages the field of view, and this camera can be optically co-axial(co-bore sighted) with the lidar transmitter 100. Camera images can alsobe used to facilitate intelligent range point selection among othertasks.

FIG. 15 shows another example of a process flow for the control circuit106 with respect to using the models to dynamically determine the shotlist for the transmitter 100. At step 1500, the laser energy model 108and mirror motion model 308 are established. This can operate like step1300 discussed above. At step 1502, the model parameters are updatedusing pulse return statistics (which may be derived from point cloud1404 or other information provided by the receiver 1400) and mirror scanposition feedback (e.g., from feedback system 850). At step 1504, themodels are coupled so that shot angles are assigned to time slotsaccording to the mirror motion model 308 for which shot energies can bepredicted according to the laser energy model 108. These coupled modelscan then be embedded in the shot scheduling logic used by controlcircuit 106. At step 1506, a list of range points to be targeted withlaser pulses 122 is received. At step 1508, a selection is made for thesearch depth that governs how far ahead the system will schedule shots.

Based on the listed range points and the defined search depth, the ordercandidates for laser pulse shots are created (step 1510). The mirrormotion model 308 can assign time slots to these order candidates asdiscussed above. At step 1512, each candidate is tested using the laserenergy model 108. This testing may also include testing based on the eyesafety model 1002 and a camera safety model. This testing can evaluatethe order candidates for compliance with criteria such as peak energyconstraints, eye safety constraints, camera safety constraints, minimumenergy thresholds, and completion times. If a valid order candidate isfound, the system can fire laser pulses in accordance with thetiming/sequencing defined by the fastest of the valid order candidates.Otherwise, the process flow can return to step 1510 to continue thesearch for a valid order candidate.

While the invention has been described above in relation to its exampleembodiments, various modifications may be made thereto that still fallwithin the invention's scope.

For example, while the example embodiments discussed above involve amirror subsystem architecture where the resonant mirror (mirror 110) isoptically upstream from the point-to-point step mirror (mirror 112), itshould be understood that a practitioner may choose to position theresonant mirror optically downstream from the point-to-point stepmirror.

As another example, while the example mirror subsystem 104 discussedabove employs mirrors 110 and 112 that scan along orthogonal axes, otherarchitectures for the mirror subsystem 104 may be used. As an example,mirrors 110 and 112 can scan along the same axis, which can then producean expanded angular range for the mirror subsystem 104 along that axisand/or expand the angular rate of change for the mirror subsystem 104along that axis. As yet another example, the mirror subsystem 104 caninclude only a single mirror (mirror 110) that scans along a first axis.If there is a need for the lidar transmitter 100 to also scan along asecond axis, the lidar transmitter 100 could be mechanically adjusted tochange its orientation (e.g., mechanically adjusting the lidartransmitter 100 as a whole to point at a new elevation while mirror 110within the lidar transmitter 100 is scanning across azimuths).

As yet another example, a practitioner may find it desirable to drivemirror 110 with a time-varying signal other than a sinusoidal controlsignal. In such a circumstance, the practitioner can adjust the mirrormotion model 308 to reflect the time-varying motion of mirror 110.

As still another example, it should be understood that the techniquesdescribed herein can be used in non-automotive applications. Forexample, a lidar system in accordance with any of the techniquesdescribed herein can be used in vehicles such as airborne vehicles,whether manned or unmanned (e.g., airplanes, drones, etc.). Furtherstill, a lidar system in accordance with any of the techniques describedherein need not be deployed in a vehicle and can be used in any lidarapplication where there is a need or desire for hyper temporal controlof laser pulses and associated lidar processing.

These and other modifications to the invention will be recognizable uponreview of the teachings herein.

1. A lidar apparatus comprising: a laser source; a mirror subsystemcomprising a mirror that is scannable through a plurality of scan anglesto define where the lidar apparatus is aimed within a field of view, andwherein the mirror subsystem is optically downstream from the lasersource; and a control circuit that dynamically controls a variable ratefiring of laser pulse shots by the laser source toward a plurality ofrange points in the field of view, wherein the control circuitdynamically controls the variable rate firing using (1) a laser energymodel that models how much energy is available from the laser source forlaser pulse shots over time as compared to a plurality of energyrequirements relating to the targeted laser pulse shots and (2) a mirrormotion model that models the scan angles for the scannable mirror overtime as a plurality of corresponding time slots, wherein the fired laserpulse shots are transmitted from the laser source into the field of viewtoward the range points via the scannable mirror; and wherein thecontrol circuit schedules the targeted laser pulse shots into time slotsbased on the laser energy model by evaluating different assignments ofthe targeted laser pulse shots into time slots according to the mirrormotion model to identify (i) which of the assignments complies with theenergy requirements for the targeted laser pulse shots according to thelaser energy model and (ii) which of one or more compliant assignmentsexhibits a shortest completion time according to the mirror motionmodel, wherein the compliant assignment that exhibits the shortestcompletion time defines a sequence for the scheduled targeted laserpulse shots.
 2. (canceled)
 3. (canceled)
 4. The apparatus of claim 1wherein the control circuit processes a group of scan angles to betargeted with laser pulse shots to schedule the targeted laser pulseshots by (1) identifying time slots corresponding to the scan angles ofthe group according to the mirror motion model and (2) identifyingdifferent sequences of the identified time slots that serve as thedifferent assignments of the targeted laser pulse shots into time slotsthat are to be evaluated according to the laser energy model.
 5. Theapparatus of claim 1 wherein the control circuit controls the variablerate firing of the targeted laser pulse shots by the laser source inaccordance with the defined sequence.
 6. (canceled)
 7. The apparatus ofclaim 1 wherein the time slots reflect time intervals in a range between5 nanoseconds and 50 nanoseconds.
 8. The apparatus of claim 1 whereinthe mirror motion model models the scan angles according to a cosineoscillation.
 9. The apparatus of claim 1 wherein the control circuitdrives the scannable mirror to scan in a resonant mode.
 10. Theapparatus of claim 9 wherein the scannable mirror comprises a firstscannable mirror that is scannable along a first axis, wherein themirror subsystem further comprises a second scannable mirror that isscannable along a second axis orthogonal to the first axis, and whereinthe first and second scannable mirrors define where the lidar apparatusis aimed within the field of view.
 11. The apparatus of claim 10 whereinthe control circuit drives the second scannable mirror to scan in apoint-to-point mode based on the range points to be targeted with thefired laser pulse shots.
 12. The apparatus of claim 1 wherein thecontrol circuit updates the mirror motion model based on feedback dataindicative of actual scan positions for the scannable mirror.
 13. Theapparatus of claim 1 wherein the control circuit comprises (1) a systemcontroller and (2) a beam scanner controller; wherein the systemcontroller (1) generates a shot list from a group of range points thatare to be targeted with laser pulse shots based on the scheduling and(2) provides the shot list to the beam scanner controller, wherein theshot list defines a schedule for laser pulse shots including the definedsequence; and wherein the beam scanner controller provides firingcommands to the laser source based on the provided shot list.
 14. Alidar apparatus comprising: a first mirror that is scannable to aplurality of first mirror scan angles; a second mirror that is scannableto a plurality of second mirror scan angles, wherein the scan angles forthe first and second mirrors define, in combination, a plurality ofrange points to which the lidar apparatus is aimed in a field of view; acontrol circuit; and a laser source that is optically upstream from thefirst and second mirrors; wherein the laser source generates laserpulses for transmission into the field of view via the first and secondmirrors in response to firing commands from the control circuit; andwherein the control circuit (1) controls scanning of the first andsecond mirrors, (2) maintains a laser energy model that dynamicallymodels available energy for laser pulses from the laser source overtime, (3) maintains a mirror motion model that models the first mirrorscan angles over time as a plurality of corresponding time slots, (4)determines available energy from the laser source for laser pulses at aplurality of the time slots based on the laser energy model, (5)determines, based on the laser energy model and the mirror motion model,a timing schedule for laser pulses to be transmitted toward a pluralityof targeted range points in the field of view, wherein the targetedrange points have corresponding first mirror scan angles, and (6)provides firing commands to the laser source based on the determinedtiming schedule to trigger generation of the laser pulses fortransmission from the laser source into the field of view toward thetargeted range points via the first and second mirrors; and wherein thecontrol circuit determines the timing schedule by (1) identifying timeslots that, according to the mirror motion model, correspond to thefirst mirror scan angles of the targeted range points with respect to aplurality of different shot order candidates, (2) evaluating a pluralityof different shot order candidates for the timing schedule based oncomparisons of a plurality of energy requirements for laser pulses withthe determined available energies to be fired at the identified timeslots for the different shot order candidates, and (3) selecting theshot order candidate for the timing schedule based on (i) which of theshot order candidates is compliant with the energy requirementsaccording to the laser energy model and (ii) which of one or morecompliant shot order candidates exhibits a shortest completion timeaccording to the mirror motion model.
 15. (canceled)
 16. (canceled) 17.(canceled)
 18. The apparatus of claim 14 wherein the time slots includetime slots for a scan of the first mirror in a first scan direction anda return scan of the first mirror in a second scan direction.
 19. Theapparatus of claim 14 wherein the energy requirements include energyrequirements that vary across different laser pulses.
 20. The apparatusof claim 14 wherein the time slots reflect intervals in a range between5 nanoseconds and 50 nanoseconds.
 21. The apparatus of claim 14 whereinthe control circuit controls the first mirror to scan through the firstmirror scan angles in a resonant mode.
 22. The apparatus of claim 14wherein the control circuit controls the first mirror to scan at afrequency in a range between 100 Hz and 20 kHz.
 23. The apparatus ofclaim 14 wherein the mirror motion model models the first mirror scanangles according to a cosine oscillation.
 24. The apparatus of claim 23wherein the first mirror scan angles correspond to tilt angles of thefirst mirror, wherein the mirror motion model models motion of the firstmirror according to a relationship of angle(t)=A(cos(2πft)), whereinangle(t) represents a tilt angle for the first mirror at time t, whereinA represents a maximum tilt angle for the first mirror, and wherein frepresents a scan frequency for the first mirror.
 25. The apparatus ofclaim 24 wherein the mirror motion model models the first mirror scanangles based on the modeled first mirror tilt angles in combination withan angle of the laser source relative to the first mirror whenpositioned at a defined first mirror tilt angle.
 26. The apparatus ofclaim 14 wherein the control circuit adjusts the mirror motion model viaa feedback loop that is based on tracked actual scan angles for thefirst mirror over time.
 27. The apparatus of claim 14 wherein thecontrol circuit comprises (1) a system controller and (2) a beam scannercontroller; wherein the system controller (1) maintains the laser energyand mirror motion models and (2) determines the timing schedule; andwherein the beam scanner controller control circuit (1) drives thescanning of the first and second mirrors and (2) generates the firingcommands.
 28. A method for controlling how a plurality of targeted laserpulse shots are transmitted from a laser source into a field of view viaa mirror that scans through a plurality of scan angles, wherein thetargeted laser pulse shots have corresponding energy requirements, themethod comprising: processing a group of the scan angles based on alaser energy model for the laser source and a mirror motion model forthe mirror to determine a schedule for a plurality of laser pulse shotswhich (1) target the scan angles of the group and (2) satisfy thecorresponding energy requirements for the targeted laser pulse shots;and firing the targeted laser pulse shots into the field of view via themirror in accordance with the determined schedule; wherein the mirrormotion model models the scan angles for the mirror over time as aplurality of corresponding time slots; and wherein the processing stepcomprises scheduling the laser pulse shots which target the scan anglesof the group into time slots based on the laser energy model byevaluating different assignments of the laser pulse shots which targetthe scan angles of the group into time slots according to the mirrormotion model to identify (i) which of the assignments complies with theenergy requirements for the laser pulse shots which target the scanangles of the group according to the laser energy model and (ii) whichof one or more compliant assignments exhibits a shortest completion timeaccording to the mirror motion model, wherein the compliant assignmentthat exhibits the shortest completion time defines the determinedschedule.
 29. The method of claim 28 wherein the processing step furthercomprises: identifying a plurality of time slots for the scan angles ofthe group based on the mirror motion model with respect to the differentassignments to be evaluated, wherein the different assignmentscorrespond to different shot order candidates for the laser pulse shotswhich target the scan angles of the group.
 30. An article of manufacturefor controlling how a plurality of targeted laser pulse shots aretransmitted from a laser source into a field of view via a mirror thatscans through a plurality of scan angles, wherein the targeted laserpulse shots have corresponding energy requirements, the article ofmanufacture comprising: machine-readable code that is resident on anon-transitory machine-readable storage medium, wherein the code definesprocessing operations to be performed by a processor to cause theprocessor to: process a group of the scan angles based on a laser energymodel for the laser source and a mirror motion model for the mirror todetermine a schedule for a plurality of laser pulse shots which (1)target the scan angles of the group and (2) satisfy the correspondingenergy requirements for the targeted laser pulse shots; and generatefiring commands for the laser source that cause the laser source to firethe targeted laser pulse shots into the field of view via the mirror inaccordance with the determined schedule; wherein the mirror motion modelmodels the scan angles for the mirror over time as a plurality ofcorresponding time slots; and wherein the machine-readable code toprocess the group of scan angles comprises machine-readable code forcausing the processor to schedule the laser pulse shots which target thescan angles of the group into time slots based on the laser energy modelby evaluating different assignments of the laser pulse shots whichtarget the scan angles of the group into time slots according to themirror motion model to identify (i) which of the assignments complieswith the energy requirements for the laser pulse shots which target thescan angles of the group according to the laser energy model and (ii)which of one or more compliant assignments exhibits a shortestcompletion time according to the mirror motion model, wherein thecompliant assignment that exhibits the shortest completion time definesthe determined schedule.
 31. The apparatus of claim 9 wherein thecontrol circuit controls the scannable mirror to scan at a frequency ina range between 10 kHz and 15 kHz.
 32. The apparatus of claim 1 whereinthe time slots reflect intervals in a range between 5 nanoseconds and 50nanoseconds.
 33. The apparatus of claim 1 wherein the energyrequirements include energy requirements that vary across differentlaser pulse shots.
 34. The apparatus of claim 1 wherein the laser energymodel quantitatively predicts available energy amounts from the lasersource for laser pulse shots over time based on a history of prior laserpulse shots.
 35. The apparatus of claim 34 wherein the laser energymodel models a retention of energy in the laser source after laser pulseshots.
 36. The apparatus of claim 14 wherein the laser energy modelquantitatively predicts available energy amounts from the laser sourcefor laser pulses over time based on a history of previously fired laserpulses.